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ABSTRACT 


This  report  describes  progress  on  a new  aoproach  for  improving  man-machine 
communication.  The  goal  of  the  work  is  to  significantly  expand  and  diversify  the 
capabilities  of  the  computer  interfaces  that  people  use.  The  approach  is  first  to  design 
computer  processes  that  can  assimilnto  particular  aspects  of  dialogue  between  people, 
then  to  transfer  these  processes  into  man-machine  communication. 

The  approach  requires  that  particular  aspects  of  the  human  ability  to  communicate 
be  selected  and  studied  in  detail.  This  report  describes  new  methods  of  data  collection 
developed  to  meet  this  need  and  tells  how  they  will  be  used. 

The  report  focuses  on  nine  phenomena  of  human  dialogue  which  have  been  selected 
from  approximately  23  phenomena  proposed  and  explored.  For  most  of  the  nine, 
explicit  observational  instructions  are  given  as  well. 
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This  report  is  a progress  report  rather  than  a report  of  completed  results.  While 
enough  has  been  accomplished  to  be  interesting,  it  is  all  tentative  and  subject  to 
change;  therefore  we  simply  exhibit  it. 

Most  of  the  report  is  a description  of  methods  which  a person,  called  an  Observer, 
can  use  to  identify  and  describe  nine  kinds  of  phenomena  which  arise  in  human  dialogue. 
The  phenomena  are; 


Repeated  Reference 
Requests 

Expression  of  Comprehension 
Similar  Expressions  - two  kinds 
Topic  Structure 

Incremental  and  Prerequisile  Content 
Correction  Actions 


The  methods  are  used  in  building  computer  programs  that  can  interpret  dialogue.  We 
expect  that  the  subprocesses  of  these  programs  will  be  able  to  improve  man-machine 
communication  when  they  are  implanted  in  task-oriented  systems. 

More  context  than  we  can  supply  here  is  needed  for  an  accurate  inlerpretation  of 
the  significance  of  the  observation  methods.  The  conlext-setting  sections  do  not  tell 
the  whole  story,  but  we  expect  lhat  most  readers  only  want  a brief  overview.  The 
report  does  not  contain  a detailed  presentation  of  the  relationship  of  this  work  to 
preceding  and  concurrent  research.  This  has  been  done  elsewhere  (Mann  1974)  and 
will  be  updated  in  forthcoming  publications  and  reports. 
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The  work  reported  here  is  not  embedded  in  a traditional,  we'l-worn  approach  to 
communication.  It  is  a new  approach  to  the  study  of  person-to-person  communication, 
with  particular  attention  to  improving  man-machine  communication  as  a consequence. 
The  report  deals  almost  exclusively  with  data-definition  and  data-collection  portions  of 
the  methodology,  which  are  but  a small  part  of  the  whole.  The  approach  draws  heavily 
on  computer  science,  linguistics,  and  psychology,  and  on  other  disciplines  to  a lesser 
extent. 

In  yet  another  way  this  report  is  not  representative  the  whole.  It  suggests  a 
particular  broad  scope  of  attention  for  which  we  are  now  prepared  to  do  certain  kinds 
of  data  development.  But  we  know  that  the  whole  scope  cannot  be  approached  at 
once;  it  must  be  done  gradually,  and  the  selectivity  of  that  movement  is  not  at  all 
apparent  in  the  report. 

Finally,  some  cautions  are  in  order  about  the  Observation  Instructions  in  the  report. 
They  are  not  really  ready  for  direct  use  by  anyone  other  than  the  authors.  In 
developing  them,  primary  attention  was  paid  to  what  kinds  of  things  to  annotate  rather 
than  to  the  notational  forms  or  the  presentation  of  ideas  to  the  Observer.  The  major 
reason  for  this  is  that  we  expect  the  substance  of  the  instructions  to  change 
significantly  under  the  pressure  of  the  next  few  months  of  use.  Any  significant 
cosmetic  or  presentational  cleanup  at  present  would  be  premature,  as  would  any 
attempt  to  determine  their  reliability. 
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As  a progress  report,  it  is  a contribution  to  empirical  research  on  human  symbolic 
communication,  and  thereby  to  man-machine  communication. 

Durng  the  past  two  years,  there  has  been  a very  substantial  increase  in  the 
number  of  on-line,  interactive  users  of  computers  in  the  military.  More  important,  many 
of  the  long-term  military  aspirations  for  command-control  and  for  administrative  and 
support  functions  call  for  computer-based,  on-line,  interactive  systems.  Thus  this 
research  is  meeting  a growing  need  for  effective,  user-oriented,  man-machine  interfaces 
directly  and  effectively  supporting  military  requirements. 
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2.  THE  RESEARCH  ENVIRONMENT 
OF  THE  OBSERVER'S  METHODS 


In  this  section  we  will  relate  the  Observer’s  task  to  our  general  methodology.  Each 
Observer  (working  with  a single  dialogue)  carries  out  one  instance  of  the  basic 
experiment,  which  in  turn  will  lead  to  a ««^*del  for  that  particular  dialogue.  This  basic 
experiment  is  to  be  repeated  many  times  with  different  Observers  and  dialogues. 
Those  dialogue-comprehension  processes  which  have  repeatedly  shown  themselves 
valuable  throughout  these  experiments  will  then  constitute  the  overall  results;  they  are 
the  primary  candidates  for  the  final  step  of  embedding  in  task-oriented  system 
applications. 

THE  BASIC  EXPERIMENT 

The  basic  experiment  consists  of  four  steps,  as  illustrated  in  Figure  2-1:  1)  acquire 
transcript  of  dialogue  for  study,  2)  gather  Observer’s  commentaries,  3)  construct  a 
process  model  to  account  for  these  observations,  and  4)  compare  the  actions  of  the 
processes  in  the  model  with  the  Observer’s  judgments  and  Observer’s  behaviors. 

Dialogue  Transcripts 

We  have  chosen  to  deal  with  dialogues  only;  we  do  not  plan  to  cover  cases  of 
multi-person  conversation.  The  dialogue  must  be  in  a machine-readable  representation. 
We  will  use  a conventionally  typed  transcript  of  the  text  of  the  dialogue,  perhaps  with 
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TRANSCRIPT 


Figure  2-1 


Basic  Dialogue  Case  Experiment 
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some  timing  information.  We  do  not  now  envision  any  attempt  to  include  and  use  such 
subtleties  as  facial  expression,  gestures,  and  intonation.  In  order  to  assure  ourselves 
that  the  transcripts  are  not  systematically  excluding  aspects  which  were  significant  in 
the  original  conversation,  we  will  examine  only  those  dialogues  which  were  originally 
conducted  over  separated  media  (e.g.  teletype,  voice  radio,  etc.)  where  the  participants 
successfully  communicated,  despite  the  lack  of  these  other,  rich  sources  of  information. 


We  currently  have  several  hundred  transcripts  of  dialogues  from  a wide  variety  of 
sources.  These  include: 


1)  Help-seeking  dialogues  from  the  TENEX  time-sharing  system. 

2)  Astronaut-and-Ground-Control  dialogues  from  the  Apollo-13 
mission. 

3)  Tutoring  transcripts  from  various  sources. 

4)  Transcripts  of  radio  talk  shows. 

We  have  been  using  some  of  these  to  pretest  the  directions  to  the  Observers.  Some  of 
these  transcripts  fail  our  criteria  in  one  or  more  ways,  often  because  of  occasional 
intrusions  by  a third  party.  We  are  also  developing  a facility  to  gather  transcripts 
ourselves,  directly  from  a terminal-to-terminal  dialogue. 

Since  there  are  certain  phenomena  we  definitely  do  not  plan  to  model,  and  since  we 
are  committed  to  giving  precisely  the  same  transcript  to  the  model  as  to  the  Observer, 
we  will  perform  a certain  amount  of  "cleaning  up"  of  the  transcript  to  get  rid  of  the 
features  which  we  know  the  model  will  not  be  able  to  deal  with.  In  particular,  we  are 
not  modeling  the  participants’  ability  to  deal  with  mispelings,  abbrev’ns,  and 
wordsruntogether,  the  various  ways  in  which  a typist  (especially  on-line  to  a computer) 
will  indicate  local  correx\Xctin\Nons,  etc.  These  and  similar  kinds  of  "noise"  will  be 
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corrected  before  either  the  model  or  the  Observer  see  them.  What  will  not  be  cleaned 
up  is  the  anomalous  grammar,  inappropriate  choice  of  words,  saying  things  which  clearly 
(to  us)  weren  t meant,  and  the  assorted  tumblings  with  words  which  happen  when  the 
speaker  can’t  find  the  right  way  to  express  himself. 

Additional  details  of  how  a transcript  is  prepared  for  the  Observer  are  given  in  the 
Appendix,  with  examples. 

Observer  Commentaries 

Having  chosen  a particular  dialogue  for  study,  we  next  give  it  to  an  Observer, 
chosen  from  outside  our  research  team,  to  obtain  his  commentary  on  the  dialogue.  This 
activity  is  described  in  detail  in  sections  5 through  12  of  the  report.  (Thus  the  report 
covers  methods  for  a part  of  the  experiment  cycle  rather  than  the  entire  cycle.) 

Dialogue  Process  Models 

With  both  the  transcript  and  the  Observer’s  annotations  to  guide  us,  the  next  step  is 
to  build  a set  of  processes  which  will  maintain  a pair  of  simulated  memories  (of  the 
information  states  of  the  two  participants)  based  on  the  transcript  as  input.  Although 
we  will  feel  free  to  reuse  processes  from  the  models  in  our  previous  experiments  when 
convenient,  our  intentions  are  that  each  of  these  models  be  a one-of-a-kind  program, 
responsible  only  for  the  one  dialogue  and  one  Observer  on  which  it  was  based.  This 
view  contrasts  with  the  more  conventional  approach  of  building  a single  system  which, 
with  each  iteration  of  the  experimental  cycle,  would  be  expected  to  adequately  model 
the  new  dialogue/commentary  pair  as  well  as  all  the  preceding  ones. 
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Obviously,  certain  supporting  processes  will  be  needed,  regardless  of  the  choice  of 
transcript.  Among  these  might  be  processes  for: 

1.  Natural  language  parsing 

2.  'Semantic  memory  management 

3.  Inference 

4.  Discrepancy  detection  and  resolution 

5.  Hypothesis  generation  and  testing 

6.  Evidence  evaluation 

7.  Awareness  of  time  passage  in  dialogue 

8.  Attention  focus 

9.  Selective  forgetting 

10.  Time  and  space  resource  allocation 

11.  Extensive  trace  and  debugging  facilities 

These  prospective  supporting  processes  share  common  technical  histories.  Each 


has  precedents  in  existing  computer  programs,  although  the  combination  does  not.  Each 
one  is  subject  to  simplification  (relative  to  past  instances)  because  our  models  are  case 
models  rather  than  general  systems.  For  each,  there  exist  adequate  technology  and 
personal  skills  for  building  the  necessary  experimental  programs.  Nearly  all  of  this 
expertise  comes  from  artificial  intelligence,  cognitive  psychology,  and  computational 
linguistics. 


: 
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In  any  model  of  a single  dialogue,  there  is  an  issue  of  what  parts  of  the  model 
nonsignificant  because  they  are  ad  hoc.  This  problem  is  addressed  explicitly  at 
multiple  experiment  level. 


are 

the 


! 


i. 


9 


- 


F 

■ 

■ 


i 


I 

I 


i 

I 


Comparison  of  Process  Behavior  with  Observer  Commentary 

Once  we  have  constructed  the  model  we  will  conduct  an  extensive  comparison  of  the 
behavior  of  the  model  with  the  Observer’s  commentary.  We  expect  to  identify  three 
sets  of  results  from  each  experiment: 

1.  Program  states  and  actions  corresponding  to  the  Observer’s  commentary. 

2.  Phenomena  recorded  in  the  Observer’s  comments  for  which  there  are  no 
corresponding  states  in  the  program. 

3.  Program  states  which  appear  to  contradict  the  Observer’s  commentary. 

In  addition,  for  those  aspects  of  the  model  which  bear  a positive  relationship  to  the 
Observer’s  commentary,  we  will  estimate  the  (actual  or  potential)  generality  of  the 
methods  involved. 

For  behavior  noted  by  the  Observer  but  contradicted  (or  ignored)  by  the  model,  we 
will  analyze  why  it  was  difficult  or  undesirable  to  achieve  accuracy  in  the  model  along 
these  dimensions. 

MU I.TI-KX PF.RIM RNT  PROCESS  EV/IUJ/ITION 

As  we  repeat  the  above  experiment  on  new  dialogues  with  new  Observers,  we 
expect  to  accumulate  a collection  of  processes  which  are  reusable  as  is,  or  with  minor 
generalization.  Those  processes  which  continue  to  prove  useful  over  a collection  of 
experiments  are  the  Ones  which  we  will  consider  candidates  for  export.  (See  Figure 
2-2) 
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Figure  2-2  - Process  Evaluation  in  Multiple  Experiments 
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This  is  the  answer  to  the  problem  of  adhocness.  Processes  are  not  identified  in 
this  vfay  unless  they  continue  to  work  over  a range  of  dialogues.  It  is  a conservative 
answer,  in  that  some  processes  that  in  fact  are  generally  applicable  may  not  be 
exercised  enough  to  be  recognized  in  this  step.  The  range  of  cases  over  which  a 
process  has  been  tested  is  always  explicit,  so  that  the  evidence  for  its  generality  is 
clear. 

One  of  the  advantages  of  this  multistage  approach  is  that  we  expect  certain 
processes  to  prove  themselves  useful  early  and  thus  to  provide  us  with  exportable 
results  well  before  the  end  of  the  project. 


DISSEMINATION  OF  DISCOVERED  ALGORITHMS 

Dissemination  of  our  results  is  uniquely  important  for  this  project.  By  ths  very 
nature  of  our  goals,  our  work  is  broadly  interdisciplinary,  with  implications  for  computer 
science,  psychology,  linguistics,  and  other  disciplines.  The  technical  dissemination  will 
include  the  novel  features  of  the  methodology  and  the  novel  result  forms  as  well  as 
specific  processes. 


Two  kinds  of  dissemination  of  results  are  planned:  one  conventional  and  one  specific 
to  this  method.  The  first  kind  of  dissemination  will  occur  through  the  usual  scientific 
channels:  site  visits,  scientific  and  popular  publication,  conference  attendance  and 
presentations,  as  well  as  exchange  of  programs  and,  perhaps,  personnel.  When  we 
have  established  a collection  of  repeatedly  effective  processes,  we  plan  to  disseminate 
these  results  by  actually  retrofitting  the  processes  to  already  existing  man-machine 
interfaces.  Of  course,  they  can  be  designed  into  new  systems  as  well. 
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3.  UNDERSTANDING  THE  OBSERVER’S  TASK 


Two  kinds  of  input  information  are  available  to  the  Observer  as  indicated  in  Figure 
3-1.  He  receives  a dialogue  transcript  and  a set  of  instructions  on  how  to  annotate  it. 
He  will  also  receive  training  and  practice  in  the  use  of  these  instructions. 

The  instructions  include  a general  orientation  to  the  experimental  goals,  the  uses  of 
the  Observer’s  work,  and  the  ground  rules  for  making  judgments;  they  also  include 
directions  for  several  specific  annotation  tasks  to  be  performed  on  copies  of  the 
transcript.  These  tasks  are  performable  independent  of  each  other  except  for  a few 
specific  sequence  requirements.  These  annotations  are  the  only  output  produced  by 
the  Observer. 


The  task  is  a blend  of  specified,  definite  steps  and  personal,  subjective  judgments. 
For  example,  we  ask  the  Observer  to  tell  where  each  participant  has  expressed 
comprehension  of  the  other’s  remarks,  but  we  do  not  tell  him  how  he  is  to  know  this. 

The  subjective  parts  of  the  Observer’s  work  are  essential  because  they  engage  his 
communication  abilities  and  methods.  Certain  methods  are  used  in  both  his  own 
performance  of  his  task  and  in  the  dialogue  participants’  performance.  It  is  these 
methods,  involving  receptive  acts  in  communication,  that  will  be  represented  in  our 
models  and  their  comparative  evaluation. 

The  skills  used  by  the  Observer  are  primarily  those  of  an  ordinary 
native-communicator  in  the  language  of  the  dialogue  being  observed  (which  always 


Dialogue  Transcript 
(multiple  copies) 


OBSERVER 


Annotated 

Transcripts 


The  Observer’s  Task 
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happens  to  be  English).  One  Observer  is  used  for  all  of  the  categories;  the  annotation 
task  is  not  split  across  Observers,  and  it  does  not  include  pooled  final  judgments.  This 
assures  that  the  view  of  a dialogue  that  gets  built  into  a model  can  be  as  coherent  as 
the  individual  Observer  was.  (It  avoids  models  resembling  committee  reports.)  The 
Observer  makes  judgments  that  to  him  are  obvious,  clear  case  judgments,  so  that  the 
adequacy  of  the  evidence  in  the  transcript  is  thereby  assured.  His  work  does  not 
require  sophisticated  technical  understanding  of  language,  and  it  does  not  include 
computer  -related  notation  in  any  way.  (In  strict  experimentation  the  Observer,  the 
model  builders,  and  the  dialogue  participants  are  alv.  iys  separate.)  We  expect  that  his 
judgments  are  representative  of  those  made  by  the  great  majority  of 
native-communicators  in  the  course  of  their  actual  communication. 

The  Observer  will  be  supplied  with  information  on  the  situation,  roles,  jargon,  and 
medium  of  the  dialogue.  The  general  rule  is  that  his  access  to  each  turn  of  the  dialogue 
should  be  essentially  as  good  as  that  of  the  participants. 

Our  policy  is  not  to  deceive  the  Observer  in  any  way.  Beside  our  preference  for 
this  approach  on  moral  grounds,  it  has  a number  of  technical  benefits  as  well.  We 
believe  that  the  Observer’s  performance  will  be  more  stable  if  he  is  not  attempting  to 
discover  the  "real"  experiment.  We  also  expect  to  disseminate  the  technology  to 
interested  volunteer  Observers,  which  requires  providing  them  with  the  best  possible 
access  to  it.  Also  specialists  who  may  serve  as  Observers  from  time  to  time.  Finally, 
effective  deceit  is  difficult  to  achieve,  and  experiments  which  rely  on  it  are  inherently 


suspect. 
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4.  PHENOMENA  FOR  OBSERVATION 


THE  OBSERVATIONS 

We  have  developed  and  pretested  instructions  for  our  Observers  to  comment  on  a 
number  of  interesting  dialogue  phenomena.  These  include  recognition  of  references 
both  to  previously  expressed  concepts  and  to  previous  text.  We  also  have  instructions 
for  certain  common  dialogue  forms,  such  as  questions  and  commands,  expression  of 
comprehension  (and  confusion),  and  the  correction  of  errors  and  misunderstandings.  We 
have  a number  of  ways  to  capture  aspects  of  the  content  conveyed  in  a dialogue  and 
the  knowledge  relied  upon  to  comprehend  this  content.  This  includes  ways  of 
generating  and  judging  paraphrases  of  dialogue  utterances  that  permit  a detailed 
examination  of  the  role  of  context  in  dialogue.  In  addition,  we  have  developed 
instructions  for  Observers  to  annotate  the  topic  structure  in  dialogue  as  introduced  and 
terminated  by  each  participant. 


CRITERIA  FOR  SELECTING  PHENOMENA 

The  various  phenomena  described  briefly  above  (and  in  detail  below)  by  no  means 
encompass  all  aspects  of  dialogue.  There  are  a number  of  criteria  that  we  have 
informally  used  to  select  phenomena  to  observe. 


i 
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1)  Importance: 

First  of  all,  we  have  considered  only  phenomena  that  we  felt  were  an 
important  component  in  dialogue.  Each  set  of  instructions  sheds  light  on  an 
aspect  of  dialogue  that  we  feel  will  be  central  to  an  interesting  model  of 
two-party  communication. 

We  have  required  that  phenomena  be  central,  in  the  sense  that  the 
communication  would  break  down  or  be  significantly  changed  in  character  if 
the  phenomenon  in  question  were  eliminated.  Phenomena  related  to  a 
particular  dialogue  source  or  context  were  not  chosen,  and  phenomena  which 
appear  substantially  unaltered  in  monologues  were  not  chosen. 

2)  Clarity: 

In  the  course  of  writing  these  instructions,  we  developed  and  modified  our 
concepts  of  the  phenomena,  since  we  were  forced  to  specify  in  some  detail 
what  we  wanted  to  observe.  The  original  five  categories  of  phenomena  to 
observe  specified  in  the  original  description  of  our  work  (Mann,  1974)  were 
developed  into  the  present  nine  categories  through  this  classification  process. 
The  instructions  in  this  report  have  typically  been  through  three  or  four  major 
revisions  to  improve  clarity.  Some  of  the  phenomena  that  we  consider 
important  will  not  be  included  in  immediately  forthcoming  experiments  because 
we  have  not  so  far  been  able  to  write  clear  instructions  for  their  observation. 

3)  Reliability: 

Finally,  the  clear  notions  of  what  to  observe  had  to  be  translated  into 
instructions  that  produced  reasonably  consistent  observations  across 
Observers  in  our  pretests.  Our  informal  feeling  of  the  present  level  of 
reliability  can  be  expressed  as  follows:  One  could  define  appropriate  measures 
of  agreement  on  these  annotations,  fairly  and  reasonably,  with  which  the 
authors  would  score  above  90  percent  on  most  of  these  categories,  working 
on  the  kind  of  dialogues  involved  in  the  pretest. 


, 
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The  Development  of  Observation  Categories 

In  the  process  of  developing  the  nine  categories  presented  in  this  report,  we 
examined  fourteen  other  observation  categories  in  detail. 


i 

} 

\ 

: 

, 


17 


Three  of  these  other  categories  were  dropped  because  they  failed  one  of  the 
criteria  described  above.  This  was  usually  because  the  concepts  could  not  be 
operationalized  into  coding  rules  for  Observers. 

In  many  other  cases,  we  were  able  to  shift  from  initially  unusable  categories  to 
others  that  bore  on  the  same  phenomena  in  a way  which  met  our  constraints.  Seven 
categories  developed  into  other  categories  in  this  way. 

Finally,  four  categories  of  observation  are  still  in  a state  of  development.  These 
categories  are  promising,  but  have  not  ',et  developed  to  the  stage  where  they  can  be 
presented  here.  These  include  observations  of  ceremonies  that  occur  in  dialogues,  of 
comments  about  the  dialogue  itself,  of  the  speaker’s  functions  (or  purposes)  in 
generating  his  utterances,  and  of  defining  events  in  which  the  meanings  of  new  terms  or 
concepts  are  explicitly  given. 

So  the  set  of  observation  categories  described  in  this  report  are  those  members  of 
a larger  set  that  survived  the  criteria  above.  We  would  expect  to  add  to  this  set  in  the 
future  after  experience  with  modeling  and  observation  brings  new  aspects  of  dialogue 
to  light. 

POST- PROCESS  INC 

As  described  previously,  the  commentary  of  the  Observers  will  be  used  to  test  the 
dialogue  models  we  build.  This  makes  it  necessary  to  compare  the  commentary  to  the 
behavioi  ■ the  model.  In  some  cases,  this  comparison  is  straightforward.  For  example, 
when  the  Observer  annotates  that  two  sets  of  words  refer  to  the  same  concept,  it  will 


Urn, 


••  - 


IS 


be  easy  to  tell  whether  the  model  also  creates  this  correspondence.  In  other  cases, 
some  post-processing  of  the  Observer’s  annotation  will  be  required  before  we  can  make 
a comparison.  For  example,  we  have  derived  a simple  algorithm  for  specifying  the 
areas  in  which  a topic  is  shared  by  both  participants,  given  the  annotation  of  topic 
initiation  and  termination  for  each  participant  separately.  We  expect  to  develop  further 
post-processing  algorithms  to  enable  uc  to  test  parts  of  the  model  for  which  we  have 
no  direct  observation  methods. 

FORM /IT  OF  THE  INDIVIDUAL  INSTRUCTIONS  SECTIONS 

The  next  sections  will  discuss  the  instructions  developed  so  far.  For  each  set,  we 
will  present  the  phenomena  to  be  observed,  the  instructions  themselves,  and  an  example 
of  applying  the  instructions  to  a real  dialogue. 

The  dialogue  used  for  each  example  is  an  interaction  between  a computer  user  and 


the  computer  operator,  conducted  remotely  via  computer  terminals.  Thus  the 
transcript,  which  is  given  in  full  in  the  Appendix,  captures  almost  all  the  interaction 
(only  the  timing  specifications  are  lacking). 
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5.  ORIENT AT/ON  FOR  THE  OBSERVER 


The  Observers’  directions  are  presented  in  several  sections,  one  for  each  of  the 
classes  of  phenomena  we  are  interested  in  having  annotated.  There  is  a small  set  of 
instructions  which  precede  and  are  common  to  each  of  these  sections.  For  the  sake  of 
brevity  these  are  presented  here,  once,  rather  than  at  the  head  of  each  separate  set. 


OBSERVER'S  GENERAL  INSTRUCTIONS 

We  are  interested  in  your  commentary  on  those  features  and  phenomena 
which  you  regard  as  clear  and  obvious.  Do  not  worry  about  making  subtle  or 
questionable  judgments.  For  those  instructions  of  the  form:  "Identify  those 
parts  of  the  dialogue  having  the  property  ...  ",  if  you  have  any  doubts 
whether  a certain  segment  meets  the  criteria  set  forth  in  the  directions,  just 
ignore  it.  Some  instructions  will  have  the  form:  "For  a piece  of  dialogue 
already  identified,  classify  it  into  One  of  the  following  categories:  ...  ".  These 
sets  of  categories  will  always  have  one  labelled:  "other/unknown/unsure";  use 
this  classification  freely  if  you  aren’t  sure  that  one  of  the  other  classifications 
is  appropriate. 

Some  of  the  transcripts  we  are  working  with  have  been  typed  by  a 
secretary,  listening  to  a recorded  dialogue;  others  have  been  typed  by  the 
participants  themselves  as  they  were  conducting  the  dialogue  via  terminals. 
You  will  be  told  which  of  these  cases  applies  to  the  dialogue  you  are 
annotating.  This  is  only  important  to  you  in  one  regard:  punctuation.  If  the 
transcript  was  prepared  by  anyone  other  than  the  actual  participant  in  the 
dialogue,  then  the  punctuation  you  see  is  to  be  taken  as  only  a "good  guess," 
possibly  in  error.  On  the  other  hand,  if  the  transcript  was  created  by  the 
participants’  typing,  then  the  punctuation  can  be  taken  to  be  what  thr 
participant  actually  expressed,  with  no  editorial  interpretation  involved. 

Although  a few  of  the  directions  ask  you  to  differentiate  between  the 
knowledge  of  the  two  participants,  most  do  not,  since  it  is  assumed  that  both 
participants  understood  the  conversation  in  substantially  the  same  way.  In 
the  case  where  the  instructions  clearly  presume  that  the  participants  share  an 
understanding  of  what  is  being  communicated  and  you  perceive  that  any  of 
your  annotations  reflect  only  the  understanding  of  onr  of  the  participants, 
indicate  this  by  adding  the  name  (letter)  of  that  participant  to  the  annotation. 


t 


A number  of  the  directions  call  for  you  to  indicate  a certain  segment  of 
the  dialogue  having  some  property.  You  may  find  that  the  segment  has  sharp 
boundaries,  or  they  may  be  somewhat  indefinite.  For  all  these  directions 
which  call  for  you  to  mark  these  segment  boundaries  (with  what  we  will  call 
"segment  markers")  the  following  general  instructions  apply: 

Put  a ”["  at  the  point  where  a segment  abruptly  starts,  and  put  a "]"  at  the 
point  where  a segment  suddenly  ends. 

Graphically: 


A:  text  text  text 

<<<<<<<<<<<<<<< 
(definitely 
NOT  part 
of  segment) 


[ text  text  text 

>>>>>>>>>>>>>> 
(definitely 
IS  part 
of  segment) 


After  you  finish  marking  these  abrupt  boundaries,  go  back  to  the 
beginning  of  the  dialogue  and  determine  where  gradual  starts  and  ends  occur. 
Mark  a "("  at  the  first  point  that  a segment  is  clearly  under  way,  and  a "(="  at 
the  latest  point  (prior  to  the  "("  ) that  clearly  is  not  included  in  the  segment. 
Similarly  for  gradual  segment  ends,  mark  a ")"  at  the  last  point  that  the 
segment  is  clearly  under  way,  and  a "=)"  at  the  first  point  (following  the  ”)”  ) 
that  the  segment  clearly  does  not  include. 

Graphically: 


B:  text  text  text  text  ) text  text  text  text  = ) text  text  text 

<<<<<<<<<<<<<<<<<<<<I  I >>>>>>>>>>>>>>>>>>>> 

(definitely  IS  part  I (unclear  whether  I (definitely  not 

of  segment)  I or  not  is  part  I part  of  segment) 

I of  segment)  I 

I I 

Notice  that  each  use  of  "["  is  equivalent  to  a simultaneous  occurrence  of 
"(»"  and  "(". 

Several  of  the  instructions  will  call  for  you  to  create  labels  for  segments 
of  the  dialogue;  these  labels  will  consist  of  one  or  more  letters  followed  by  a 
number.  This  choice  of  numbers  is  completely  arbitrary  --  you  need  not  be 
concerned  with  keeping  them  either  consecutive  or  in  order  (or  corresponding 
to  the  numbers  used  in  compliance  with  other  sets  of  directions). 


K ff v 'IIIIV  1 
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Whenever  you  feel  that  the  actual  annotations  are  not  capturing  the 
phenomena  to  which  they  are  addressed,  please  note  this  and  add  your  own 
comments  at  any  point. 


! 


6.  REPEATED  REFERENCE  AND  TEXT  REFERENCE 


One  of  the  most  common  phenomena  in  language  is  that  of  reference,  the  use  of  a 
sequence  of  words  by  a speaker  to  refer  to  an  object  or  concept.  Taken  as  a whole, 
the  concept  of  reference  is  extremely  complicated,  encompassing  as  it  does  almost  the 
entire  range  of  human  conceptual  and  experiential  abilities.  Not  surprisingly,  we  do  not 
aspire  to  model  the  full  range  of  reference  phenomena  in  natural  language. 

We  have  made  two  major  restrictions  on  the  phenomena  we  are  investigating:  First, 
we  are  only  interested  in  those  cases  for  which  the  same  object  or  concept  is  referred 
to  by  a participant  in  more  than  one  place  in  the  dialogue.  Second,  we  do  not  attempt 
to  pin  down  exactly  what  is  being  referred  to— rather,  we  simply  want  to  determine 
which  sequences  of  words  have  been  used  to  refer  to  the  same  thing  (whatever  it  may 


Within  these  boundaries,  we  investigate  a few  of  different  flavors  of  reference: 


1.  Two  regions  of  dialogue  refer  to  the  same  thing. 

2.  One  region  refers  to  a set  of  things  and  a second  region  refers  to  a single  thing 
which  is  a member  of  that  set. 

3.  One  region  refers  to  a set  of  things,  a subset  of  which  is  referred  to  by  the 
second  region. 

We  are  also  investigating  a form  of  reference  with  a somewhat  different  thrust:  a 


so-called  Text  Reference,"  made  to  a string  of  words  in  the  preceding  dialogue  itself 
(and  not  to  the  referent  of  that  preceding  string  of  words!). 


I 

I 
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OBSERVER'S  DIRECTIONS  FOR 

REPEATED  REFERENCE  AND  TEXT  REFERENCE 


Repeated  Reference 


A Repealed  Reference  is  said  to  occur  whenever  two  phrases  in  a dialogue 

are  used  to  refer  to  the  same  thing  (object,  person,  activity,  concept,  ...). 

Some  examples  of  such  Repealed  References  are  the  following: 

1.  Repeated  Reference  to  an  object 
A:  I really  like  your  wrist  watch. 

B:  It  only  cost  S50.00. 

2 Repeated  Reference  to  a person: 

•V  I spoke  to  Max’s  sister  yesterday. 

B:  Did  you  know  that  she’s  going  to  Europe 
next  month? 

3.  Repeated  Reference  to  an  activity 
A:  Sky  diving  is  a great  sport. 

B:  Yes,  and  it’s  not  as  dangerous  as  some 
people  think. 

A.  Repeated  Reference  to  a concept 

A:  Jim  told  me  that  jobs  are  very  hard  to 
find  now. 

B:  Yes,  that’s  true.  It  certainly 
is  sad... 


The  two  phrases  may  be  identical  in  wording,  but  in  general  are  not.  A 
Repeated  Reference  may  be  effected  by  as  little  as  a single  word,  or  as  much 
as  a complete  utterance.  A phrase  may  participate  in  many  Repeated 
References  or  in  none. 


Whenever  in  your  judgment  two  phrases  are  obviously  used  to  refer  to 
the  same  thing  1)  underline  each  phrase,  2)  assign  the  first  a unique  number, 
3)  write  this  number  in  either  margin,  opposite  each  of  the  two  phrases,  A) 
connect  each  phrase  to  the  number  you  just  wrote  with  a line. 


U:  Is  [NAME5]  still  around? 


■(i) 


0:  No  he  isn’t. 

(I)~T 


If  the  same  tiling  is  referred  to  by  means  of  more  than  two  phrases,  ise 
the  number  assigned  to  the  first  such  phrase  in  annotating  each  subsequent 
Repeated  Reference. 


a~,„. 


'V  * 1 
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U:  Can  you  recover  those  files  for  me? 


(7) 


7 


0:  OK  hold  on  just  a min.  and  I’ll  try  to  find  them 

X 


—(7) 


0:  OK  yes  I have  found  the  files  you  were  concerned  about 


(7) 


7 


You  may  discover  that  one  phrase  is  used  to  refer  to  a single  element  of 
the  set  or  collection  of  things  referred  to  elsewhere  by  means  of  a second 
phrase.  Whenever  this  happens,  annotate  these  as  Repeated  References  with 
the  addition  of  an  "E"  or  "S"  (respectively)  on  the  line  from  the  Reference  to 
an  element  or  to  a set.  For  example: 

0:  I am  really  not  sure,  however  I know  that  that  would 

l 1 <r«.) 

not  be  responsibility  we  would  take  on  ourselves  ... 

-I -J 

Sometimes  one  may  find  that  one  phrase  is  used  to  refer  to  a set  and  a 
second  phrase  is  used  to  refer,  not  to  an  element,  but  to  a subset  of  the  set 
referred  to  by  means  of  the  first  phrase.  In  such  cases  of  Repeated 
References,  mark  the  Reference  to  the  set  with  an  "S"  (as  requested  above), 
and  mark  the  Reference  to  the  subset  with  "SS".  For  example: 

U:  Are  thejiles  that  are  on  disk  archived  every  week? 


(3  X)S- 


J 


0:  Yes. 


U:  I had  ^omefile^  in  one  day  and  the  next  they  were  gone 


I 


tney 


ss(  37.) 


In  the  case  of  one  Reference  within  another,  underline  and  bracket  the 
larger  one;  overline  and  bracket  the  smaller  one.  For  example: 


S(3) 


c 


U:  is  missing. 

<3)«T  — 


f 


O:  Was  jt  marked  to  be  archived? 

(3)£ — 

f 

U:  Nol  none  of  /therrl  Jwere. 

rz_ 


-S(3) 

$5(3) 


Referring  expressions  are  a very  common  phenomenon  in  language — so 
much  so  that  some  are  easily  overlooked  To  help  you  be  sure  that  you  are 
considering  all  possible  Repented  iu  i -rene.es,  pay  particular  attention  to 
phrases  beginning  with 

1.  Personal  Pronouns 

(I  you  he  she  it  we  they  me  him  her  us  them)  N.B.:  these  will  be 
one-word  phrases. 

2.  Quantifiers 

(a  an  the  that  each  all  some  any  every  one  ...) 

3.  Pronominal  Possessives 

(my  your  his  her  our  its  the;'  mine  yours  hers  ours  theirs)  N.B.: 
these  phrases  may  have  two  Repeated  References : one  for  the 
possessor  and  one  for  the  object  possessed. 

4.  Wh-words 

(who  what  when  where  why  which  how  ...) 

These  words  are  intended  to  serve  as  clues  to  most  of  the  potential 
Repeated  References  but  are  not  intended  to  be  comprehensive.  ANY  two  sets 
of  words  which  you  feel  refer  to  the  same  thing  are  to  be  treated  as  a 
Repealed  Reference. 


Since  they  are  so  common  in  dialogue,  we  are  making  a special  case  of 
first  and  second  person  pronouns.  All  occurrences  of  first  and  second  person 


......  .. ....... 
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SINGULAR  pronouns  (I  me  my  mine  you  your  yours),  when  used  to  refer  to  self 
and  partner,  respectively,  are  simply  to  be  underlined,  with  no  further 
annotation  necessary.  Should  you  find  an  occurrence  of  a plural  pronoun  in 
the  first  or  second  person,  select  one  convenient  occurrence  of  a singular 
pronoun  used  by  (he  same  speaker,  in  the  same  person.  Annotate  these  two 
(the  occurrences  of  the  singular  and  plural  pronouns)  as  per  the  instructions 
for  general  "element"  and  "set"  References,  above.  Note  that  first  ana  second 
person  singular  possessive  pronouns  may  still  require  annotation  as  a 
Reference  to  the  thing  possessed,  but  not  as  a Reference  to  the  possessor. 

You  should  be  careful  to  distinguish  (where  possible)  between  "you" 
referring  to  the  speaker’s  partner  and  "you"  referring  to  an  indefinite  "other" 
(e.g.,  as  in  "You  can’t  fight  City  Hall").  When  you  come  across  a clear  instance 
of  an  impersonal  or  general  "you",  do  not  underline  it;  rather,  indicate  the 
occurrence  of  an  impersonal  "you"  by  circling  it  in  the  dialogue.  If  you  are 
not  sure  whether  or  not  a particular  occurrence  of  "you"  is  impersonal,  simply 
do  not  annotate  it  at  all  - neither  underline  it  nor  circle  it. 
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Text  Reference 

A Text  Reference  occurs  whenever  reference  is  made  to  previously 
occurring  words  within  the  transcript.  For  example,  in  the  sentence 


Go  3 blocks  and  turn  North; 

By  North  I mean  towards  the  mountains. 


the  second  use  of  "North"  is  a Text  Reference  to  the  first.  We  call  this  a 


i 

i 


1 
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Text  Reference,  because  it  refers  to  the  previous  use  of  the  word  itself,  rather 
than  to  its  meaning. 

Many  of  what  we  are  calling  Text  References  would  be  indicated,  in  a 
formal  publication,  by  quotation  marks.  However,  the  transcripts  typically  will 
not  include  these  quotation  marks.  Note  that  a Text  References  need  not 
repeat  the  words  to  which  reference  is  being  made.  For  example: 

In  your  last  three  sentences,  you  failed  to  answer  me. 

Mark  Text  References  in  the  same  manner  as  Repented  References  , 
and  distinguish  them  with  a "TR"  next  to  the  number.  For  example: 

O’! )rn  — 

U:  The  names  are  | [f ile  namel].dat;l  ^ ABDLLLL  ? J check  that 

V. r<  U*> 

[file  namel].dat;l,2  [file  name2].f4il  and  [file  namr3],f4;l 


0:  On  the  first  one  I assume 


the  ABDLLLL 


was  an  error  right? 

— T(l  (3L3) 


U:  Right  that  should  have  been  [file  name].dat;  1 and  2. 


Sometimes  there  will  be  a Text  Reference  to  all  or  one  of  a number  of 
separate  pieces  of  text,  each  involving  the  same  phrase  or  word.  You  need 
annotate  only  the  most  recent,  prior  occurrence  of  the  text  which  seems  to 


have  the  same  meaning.  For  example: 


Be  sure  you  examine  all  occurrences  of: 

SAY 
MEAN 
MENTION 
TALK  ABOUT 
DEFINE 

and  similar  phrases  (and  their  other  tenses)  to  see  whether  they  signal  the 
presence  of  Text  Reference* . As  before,  this  list  is  is  only  suggestive  and  is 
not  presented  as  being  comprehensive. 


Figure  6-1  represents  a portion  of  a dialogue  fully  annotated  for  Repented 


Reference. 
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15  0: YES 


CSV 


(.i  )■ 


16  U:  INfONE  OF  MY  DIRECTORIES^  HAD  SOME  FILES  IN  ONE  DAY  AND  THE  NEXT  [HEY  WERE 


17  GONE  AND  THEY  WERE  NOT  ARCHIVED 


18  0:  WELL, AM  REALLY  NOT  THAT  SURE  OF  HOW  THE  ARCHIVEING  OF 

19  THE  FILES  ARE  DONE  S0_[ WOULD  SUGGEST  THAT  YOL[SEND  A MESSAGE  TO 

20  [name2]  IN  REGARDS  TO  THIS  PROBLEM.  THE  ARCHIVEING  IS  DONE 

— — 

21  DURING  THE  SWING  SHIFT  SO  IN  ORDER  THAT  YOU  DO  DON'T  NOT  GET 

22  MISLED  BY  SOMETHING_l_AM  NOT  CERTAIN  ONJJWOULD  RATHER 

23  122  DISCUSS  THIS  WITH  [nan,e2].  HE  IS  OUT  OF  Hj^OFFICE  AT  THIS 

24  TIME  BUT  YOIJ  COULD  SEND  HIM  A MESSAGE  AND  I AM  SURE  HE  WILL  GIVE 

— □= ZL  T 

25  YOU  ALL  THE  INFORMATION  YOU  WOULD  LIKE,  [operator’s  name] 


CV)S  I26  U:  QK  [operalor’s  name],  CAN  YOU  PEOPLE/GO  INTO  ANY  DIRECTORY  AND  DELETE  FILES  WHEN  YOU 


C7), 
C 2) 


27  /NOTICE  THAT  [HEY  ARE  NOT  BEING  USFdI  [user's  name] 


(S")S 


28  0:1  AM  REAILY  NOT  SURE,  HOWEVER j_KNOW  THAT  THAT  WOULD  NOT  BE 


29  RESPONSIBILITY  WE  WOULD  TAKE  UPON  OURSELVES  TO  JUDGE  WHETHER  OR 

30  NOT  YOU  WANT  YOUR  FILES  OR  NOT. 


S(v) 

(SJ 

C7) 


31  [outside  interruption  here] 

32  0:  [user’s  name],  ARE  YOU  THERE? 


Figure  6-la  - Dialogue  Annotated  for  Repeated  Reference  - Part  a 
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33  LINK  FROM  [user’s  name],  JOB  21,  TTY  22 


34  U:  SORRY  [operator’s  name] 

35  WE  GOT  DISCONNECTED....CAN 

jr 


YOU  REtjoV 


OVER  THOSE  FILES  FOR  ME.. 

rr 


^ ^ 36  AS  FAR  AS  I KNOW  THEY  WERE  li/  THE  DIRECTORY  ON  THE  16TH...THE  NAMES 

(Srt = *=. 


37  ARE  ...[file  name  1 ].DAT j 1 ABDLLLL  CHECK  THAT  ...[file  namel].DAT;l,2 

38  [file  name2].F4;l....AND  [fil»»  "ame3].F4;l [user’s  name] 


T 


I 


r 


Eflb) 

ecv) 


-CO 


sO  o) 


eoo) 


39  0:  OK  HOLD  ON  JUST  A MINUTE  AND  I WILL  TRY  TO  FIND  THEM 


Figure  6- lb  - Dialogue  Annotated  for  Repeated  Reference  - Part  b 
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7.  REQUESTS 


. 


7 

* 


In  the  course  of  a dialogue,  frequently  one  of  the  participants  will  communicate  to 
his  partner  an  expectation  about  the  partner’s  subsequent  behavior.  We  intend  to 
include  all  such  communications  under  the  heading  of  Requests  The  Observer  is  asked 
to  detect  any  occurrence  of  a speaker’s  indicating  such  an  expectation,  and  to  classify 
each  of  these  utterances  into  one  of  five  categories: 


1.  Questions  — request  immediate,  verbal  response. 

2.  Orders  — request  immediate,  nonverbal  response. 

3.  Directives  — request  certain  behavior  in  the  future. 

4.  Rhetoricals  — look  like  Requests  but  are  not. 

5.  Prohibitives  — request  TO  NOT  DO  something 

Having  done  this,  we  ask  the  Observer  to  annotate  the  immediately  following  verbal 
response  by  the  requester’s  partner.  These  annotations  attempt  to  characterize  the 
cooperative  ot  uncooperative  nature  of  this  response. 


OBSERVER'S  INSTRUCTIONS  I’OR  REQUESTS 

We  are  interested  in  the  general  category  of  utterances  which  we  call 
Requests,  by  which  a speaker  communicates  a specific  expectation,  request,  or 
demand  to  the  hearer.  We  are  also  interested  in  the  cooperative  or 
uncooperative  nature  of  the  hearer’s  next  reply. 

There  are  many  different  ways  a speaker  may  express  a Request. 
direct:  "Please  pass  the  salt.” 

indirect:  "I  could  sure  use  some  salt." 

The  speaker  may  Request  behavior  which  is: 

"Is  MTAO  write-enabled?" 

"Please  mount  JMlla  on  DTA3." 

"Do  you  have  the  time  to  retrieve  my 
file  X from  archives?" 
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verbal: 

nonverbal: 

both: 
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The  behavior  Requested  may  be: 

immediate:  "Attach  the  pump  to  the  platform." 

delayed:  "Please  call  me  when  you  arrive." 

There  is  a set  of  utterances  which  look  like  Requests,  but  which  both 
speaker  and  hearer  know  are  not.  The  following  are  examples  of  this 
category  of  Requests: 

"Why  don’t  you  go  jump  in  the  lake?" 

"Who  do  you  think  you  are?" 

"Is  the  sky  blue?" 

"Go  fight  City  Hall!" 

Finally,  there  are  Requests  to  not  do  or  say  something: 

"Don’t  think  about  elephants." 

"Whatever  you  do,  please,  don’t  throw  me  into  the  briar  patch! 

"Don’t  put  beans  in  your  ears." 

"Never  tow  your  car  while  it  is  in  gear." 

We  divide  the  set  of  possible  Requests  into  the  following  groups: 

1.  QUESTIONS.  For  our  purposes,  "Question"  refers  to  a much  larger 
class  of  utterances  than  does  the  conventional,  casual  use  of  the  word.  By 
"Question"  we  mean  any  utterance  by  which  the  speaker  communicates  to  the 
hearer  an  expectation,  request,  or  demand  for  specific,  immediate,  verbal 
behavior. 

Verbal:  The  anticipated  response  is  an  utterance. 

Immediate:  The  hearer  is  to  initiate  his  compliance  commenc  ng  with  his 
very  next  speech  act. 

Specific:  The  expression  is  sufficiently  detailed  that  it  is  potentially 
fulfillable  by  a single,  appropriate  response.  ("How  do  I take  a square  root?" 
is  specific;  "Help  me,  I’m  lost!"  is  not.) 


2.  ORDERS.  By  "Order"  we  mean  any  utterance  by  which  the  speaker 
communicates  to  the  hearer  an  expectation,  request,  or  demand  for  specific, 
immediate,  nonverbal  behavior. 

Immediate:  The  requested  nonverbal  behavior  is  to  commence  as  soon  as 
the  hearer  completes  the  process  of  comprehending  the  order. 

Specific:  The  speaker  communicates  the  belief  that  with  this  Order  (and 
what  preceded  it)  the  hearer  has  sufficient  details  to  enable  him  to  perform 
the  desired  behavior. 
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3.  DIRECTIVES.  By  "Directive"  we  mean  an  utterance  by  which  the 
speaker  communicates  to  the  hearer  an  expectation,  request,  or  demand  for 
any  sort  of  Delayed  behavior. 

Delayed:  the  speaker  communicates  no  sense  of  immediacy  of  response  in 
his  Request  (including,  of  course,  the  case  where  a future  time  or  condition  is 
indicated  explicitly). 

4.  RHETORICALS.  A "rhetorical"  is  an  utterance  which  has  the  form  of  a 
Request,  but  which,  in  the  context  of  its  use,  is  understood  by  all  involved  to 
represent  something  other  than  a mandate  to  the  hearer. 

5.  PROHIBlTIVES.  Any  Request  to  not  engage  in  a particular  behavior  is  a 
"Prohibitive."  Note  that  this  need  not  require  a negative  word  ("Trespassers 
will  be  eaten!").  This  class  also  includes  the  "don't  do  anythmg--don’t  go 
away--don’t  be  impatient",  implied  by  "Wait!"  and  its  related  forms. 

Frequently  a Request  will  be  taken,  by  custom,  to  mean  something 
different  from  (but  possibly  related  to)  the  literal  sense  attributable  to  it.  A 
simple  example  of  this  is  the  request  "Do  you  have  s match?"  Clearly,  those 
cases  in  which  this  is  asked  as  a simple  yes/no  question  would  be  regarded  by 
the  average  speaker  in  our  culture  as  atypical.  For  the  purposes  of  the 

above  classification,  you  will  be  asked  to  label  a Request  according  to  the 

principal  force  of  tie  utterance  — the  clearly  intended  and  recognized 

meaning  of  the  Request,  independent  of  its  surface  form.  In  the  case  of  the 

above  example,  "Do  you  have  a match?"  would  be  annotated  as  though  it  were 
"Please  give  me  a match." 

Any  Request  which  falls  into  more  than  one  of  these  categories  should  be 
annotated  separately  for  each  appropriate  category. 

Should  you  encounter  an  utterance  which  seems  to  satisfy  the  general 
definition  of  Requests  described  above,  but  which  doesn’t  fit  into  any  of  these 
subcategories,  then  underline  it  and  describe  the  new  subcategory  of  which 
you  feel  it  is  a member. 

Note  that  in  nonpathological  dialogue,  any  utterance  may  create  a general 
expectation  of  relevance,  continuity,  etc.  in  the  subsequent  reply.  The 
creation  of  this  type  of  expectation  is  not  intended  to  be  included  in  our 
definition  of  Request.  Thus,  although  in  the  sequence:  A:  "I  don’t  feel  well.",  B: 
"Santa  Monica.",  B would  not  seem  to  have  satisfied  A’s  expectations,  we  would 
not  regard  A as  having  uttered  a Request. 

You  are  to  annotate  each  of  these  categories  of  Request  separately, 
according  to  the  following  directions. 
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Part  l — Questions. 

Whenever  you  see  a Question,  enclose  it  in  angle  brackets,  mark  it  with  a 
"Q"  and  a unique  number. 
e.g.: 


tart 


^Tell  me  your  name^ 


Do  not  rely  on  the  presence  or  absence  of  question  marks  in  the 
transcript?  Is  that  clear.  Remember  that  any  contiguous  region  of  an 
utterance  that  exhibits  an  expectation  of  an  immediate,  specific,  verbal 
response  falls  into  the  category  of  a Question. 


For  each  Question  so  annotated,  examine  the  immediately  following  turn  of 
the  other  person.  We  want  you  to  separate  two  classes  of  turns:  a)  the  turn 
provides  (or  begins  to  provide)  the  anticipated  behavior,  and  b)  everything 
else.  If  this  turn  contains  one  or  more  utterances  which  constitute  some  or  all 
of  the  specifically  requested  verbal  behavior,  then  annotate  the  turn  with  the 
Question  label  and  a "+"  (e.g.:  "Q5+").  Otherwise,  annotate  the  response  with 
the  same  label  and  a (e.g.:  "Q5-"). 


Note  that  the  turn  may  begin  with  verbal  behavior  outside  that  requested, 
("Hmmm,  gee,  let  me  think,  I don’t  know,  well,  yes!,  I guess  so.").  As  long  as 
the  recipient  of  the  Question  begins  to  provide  the  requested  behavior  prior 
to  his  partner’s  next  turn,  we  want  you  to  annotate  this  as  Do  not 

attempt  to  annotate  the  intervening  behavior  with  respect  to  this  particular 
Question. 

Be  sure  you  distinguish  between  those  responses  which  actually  provide 
the  desired  behavior  and  those  which  (only)  set  in  motion  a chain  of  events 
which  is  expected  to  culminate  in  the  desired  behavior.  An  example  of  this 
would  be  a response  which  sought  to  clarify  the  question,  thus  indicating  a 
willingness  to  answer  it  (but  which  was  not  itself  initiating  the  answer). 

For  those  responses  which  are  marked  select  one  of  the  following 
descriptions  which  most  closely  captures  the  function  performed  by  that 
response;  append  the  description’s  label  to  the  annotation  (e.g.:  "Q5-A3").  If 
none  seems  to  apply,  then  invent  and  describe  your  own.  If  the  response 
serves  more  than  one  of  these  functions,  choose  the  one  indicated  first;  if  this 
distinction  is  not  clear,  indicate  all  the  apparently  simultaneous  responses  (e.g., 
"Q5-A1,A2"). 


Al.  Requests  clarification  of  meaning  of  Question 
or  definition  of  a term  in  the  Question. 

A:  Hnw  old  is  your  grandmother? 

B:  Which  one? 
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A2.  Requests  clarification  of  basis  or  cause  for 
asking  the  Question. 

A:  Where  were  you  last  night? 

8:  Why  do  you  ask? 

A3.  Challenges  a premise  or  assumption  in  the 
Question. 

A:  Why  did  you  go  to  the  party  without  me? 

B:  What  makes  you  think  I was  there? 

(Note  that  A1  through  A3  usually  create  new  Questions.) 

A 4.  Indicates  that  he  may  have  or  know  the 
information  but  refuses  to  supply  it. 

A:  Where  did  you  get  the  810,000? 

8:  I refuse  to  answer  on  the  grounds  ... 

A5.  Dismisses  Question  as  meaningless  or 
otherwise  unsuitable  for  answering. 

A:  Do  you  think  the  union  will  ask  for  more 
money9 

B:  I won’t  dignify  that  with  a reply. 

A6.  Exhibits  a promise,  willingness  or  intention 
to  reply  to  the  Question,  somehow,  after  a delay. 

A:  So  what’s  the  final  figure? 

B:  I’ll  have  that  for  you  this  afternoon. 

A7.  Disclaims  knowledge  of  the  requested 
information. 

A:  Can  you  tell  me  anything  about  LINK? 

B:  Sorry,  I’m  new  here  and  don’t  know. 

A8.  Declines  to  supply  the  information. 

A:  How  much  do  you  make? 

B:  That’s  none  of  your  business! 

A9.  Does  not  take  up  the  Question  in  any  way. 

A:  Young  man,  what  are  your  intentions  concerning 
my  daughter? 

B:  Hasn’t  the  weather  been  lovely  lately? 

Identify  the  region  of  the  transcript  which  you  used  as  the  basis  for  your 
choice  of  labelling.  Use  double  angle  brackets,  i.e.,  «,  ».  Label  each 

bracket  with  the  Question  identifier,  e.g.  (Q5). 


After  you  have  read  the  whole  transcript  and  identified  all  of  the  (Q)’s,  go 
back  and  decide,  for  each  one,  whether  the  expected  response  was  ever 
supplied  in  the  transcript.  If  so,  identify  the  region  of  the  transcript  in  which 
it  was  supplied.  Use  the  segment  markers  described  in  the  general 
instructions,  i.e.,  (=,  (,  [,  ),  =).  Label  each  marker  with  the  Question  identifier, 
e.g.,  (Q5).  If  there  are  substantial  gaps  (a  sentence  or  more)  in  the  region  in 

which  the  response  is  supplied,  use  several  sets  of  marked  brackets  rather 
than  one  set.  For  each  Quection,  label  the  rightmost  ] or  =),  with  the  word 
"Partial"  if  only  part  of  the  requested  verbal  behavior  was  exhibited. 

It  may  appear  that  the  participants  end  up  with  different  views  on 
whether  the  requested  behavior  was  supplied,  or  whether  it  was  supplied  in 
full.  If  so,  then  label  your  brackets  with  the  participant’s  identifier  when 
indicating  where  the  information  was  supplied,  e.g.,  ](Q5  for  George).  Use 

two  sets  of  l rackets  if  necessary. 


Part  2 — Orders. 

A contiguous  region  of  an  utterance  that  exhibits  an  expectation  of  an 
immediate,  specific,  nonverbal  response,  falls  into  the  category  of  an  Order. 
Whenever  you  see  an  Order,  enclose  it  in  ang^  brackets  and  mark  it  with  "0" 
and  a unique  number,  as  above. 

For  each  Order  so  annotated,  examine  the  immediately  following  turn.  We 
want  you  to  separate  two  classes  of  responses:  a)  compliant  and  b)  everything 
else.  There  are  three  kinds  of  compliant  responses: 

Cl.  Asserts  completion  of  requested  activity. 

A:  Now  attach  the  second  bolt. 

B:  I’ve  already  attached  all  of  them. 

C2.  Asserts  current  performance  of  requested  activity. 

A:  Next,  empty  the  number  two  tank. 

B:  I’m  already  doing  that. 

C3.  Asserts  willingness,  ability,  and  intention 
to  initiate  requested  activity  forthwith. 

A:  Private,  do  you  think  you  could  find  me  a 
cup  of  coffee? 

B:  Yes  sir!  Right  away  sir! 

Label  the  compliant  response  with  the  Order  label  and  a "+"  followed  by 
the  type  of  response  (e.g.:  "(05+C2)").  If  you  feel  that  there  has  occurred  a 
compliant  response  which  does  not  fall  into  one  of  the  above  categories,  feel 
free  to  invent  a new  category  of  your  own.  For  those  responses  which  are 
other  than  compliant,  annotate  them  with  the  Order  label  and  a as  above. 
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For  those  responses  which  are  marked  select  one  of  the  following 
descriptions  which  most  closely  captures  the  function  performed  by  that 
response;  append  the  description’s  label  to  the  annotation  (e.g.:"(05-R3)"). 
If  none  seems  to  apply,  then  invent  and  describe  your  own. 

Rl.  Requests  clarification  of  meaning  of 
Order. 

A:  Position  the  drain  towards  the  front. 

B:  Which  side  is  the  front? 

R2.  Requests  clarification  of  basis  or  cause  for 
giving  the  Order. 

A:  Glue  the  bottom  before  the  top. 

B:  What  will  that  buy  us? 

R3.  Challenges  a premise  or  assumption  in  the 
Order. 

A:  Set  up  a meeting  with  Jones  this 
afternoon. 

B:  Have  you  forgotten?  He’s  on  vacation. 

(Note  that  A1  through  A3  usually  create  new  Questions.) 

R4.  indicates  that  he  may  be  able  to  perform  the 
Order  but  refuses  to  perform  it. 

A:  Loan  me  S100  until  payday,  if  you  have  it. 

B:  I could  find  it,  but  I don’t  lend  money  to 
anyone. 

R5.  Dismisses  Order  as  meaningless  or  otherwise 
unsuitable  for  performance. 

A:  Will  you  give  me  a ride  to  the  station? 

B:  Sorry  I can’t,  my  car’s  in  the  shop. 

R6.  Exhibits  a promise,  willingness  or  intention 
to  perform  the  Order  after  a delay. 

A:  Johnny,  go  cut  the  lawn. 

B:  I’ll  do  it  this  afternoon. 

R7.  Disclaims  ability  to  perform  the  Order. 

A:  Please  mount  my  tape  on  drive  1. 

B:  Sorry,  I’m  new  — don’t  know  how. 

R8.  Declines  to  perform  the  Order. 

A:  Subscribe  now  and  save  207! 

B:  No  thanks,  I’m  really  not  interested. 
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R9.  Does  not  take  up  the  Order  in  any  way. 

A:  Point  the  camera  towards  the  meter. 

B:  How  many  washers  are  there  supposed  to  be? 

As  above,  identify  the  region  of  the  transcript  which  you  used  as  the  basis 
for  your  choice  of  labelling.  Use  double  angle  brackets,  i.e.,  «,  ».  Label 
each  bracket  with  the  Order  identifier,  e.g.,  (05). 

After  you  have  read  the  whole  transcript  and  identified  all  of  the  (0)’s,  go 
back  and  decide,  for  each  one,  whether  it  was  responded  to  anywhere  else  in 
the  transcript.  If  so,  identify  the  region  of  the  transcript  which  constituted 
the  response.  Use  the  segment  markers  described  in  the  general  instructions, 
'-e*»  (">  [»  ]» ).  ■)•  Label  each  marker  with  the  Order  identifier,  e.g.,  (05). 

For  each  Order,  label  the  rightmost  ] or  -),  with  the  word  "Partial"  if  only 
part  of  the  requested  behavior  seems  to  have  been  provided.  If  he  tried  but 
did  not  succeed,  that’s  "Partial,"  too. 


Part  3 — Directives. 

Any  contiguous  region  of  an  utterance  which  exhibits  an  expectation  for 
specific  behavior  in  the  future  falls  into  the  category  of  a Directive. 
Whenever  you  see  a Directive,  enclose  it  with  angle  brackets,  mark  it  with  a 
"D"  and  a unique  number,  as  above. 

A Directive  has  the  sense  "Perform  behavior  X at  some  time,  or  under 
some  circumstance,  in  the  future."  Treat  each  Directive  as  though  it  said  "I 
would  like  for  you  to  perform  behavior  X ...  . Do  you  assent/commit,  now, 
to  behaving  as  indicated,  at  the  appointed  time  or  under  the  requested 
circumstances?" 

Thus,  a Directive  combines  the  functions  of  a Question  ("Tell  me  that  you 
will  ...  ) with  those  of  an  Order  {"  ...  perform  a certain  behavior.").  With 

this  in  mind,  the  response  for  each  Directive  is  to  be  annotated  according  to 
the  directions  for  Questions  (i.e.,  responding  to  a request  for  an  immediate, 
verbal  act).  The  exception  to  this  is  whenever  the  response  is  like  the  Cl, 
C2,  or  C3  responses  to  Orders.  Treat  the  requested  action  in  the  present 
tense.  In  this  case,  annotate  as  i "+"  response  to  a Directive. 

As  above,  identify  the  region  of  the  transcript  which  you  used  as  the  basis 
for  your  choice  of  labelling.  Use  double  angle  brackets,  i.e.,  «,  ».  Label 
each  bracket  with  the  Directive  identifier,  e.g.,  (D5). 


After  you  have  read  the  whole  transcript  and  identified  all  of  the  (D)’s,  go 
back  and  decide,  for  each  one,  whether  it  was  responded  to  anywhere  else  in 
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the  transcript.  If  so,  identify  the  region  of  the  transcript  which  constituted 
the  response.  Use  the  segment  markers  described  in  the  general  instructions, 
i-e->  (“i  (»  [»  ],  ),  -).  Label  each  marker  with  the  Directive  identifier,  e.g.,  (D5). 

For  each  Directive,  label  the  rightmost  ] or  =),  with  the  word  "Partial"  if 
only  part  of  the  requested  behavior  seems  to  have  been  provided.  If  he  tried 
but  did  not  succeed,  that’s  "Partial",  too. 

Parts  4 and  5 — Rhetoricals  and  Prohihitives. 

Whenever  you  see  a Rhetorical  or  a Prohibitive,  enclose  it  in  angle 
brackets  and  mark  it  with  a "R"  or  "P",  respectively,  and  a unique  number,  as 
above.  Do  not  attempt  to  annotate  the  immediate  response  to  these  Requests. 

After  you  have  read  the  whole  transcript  and  identified  all  of  the  R’s  and 
Ps,  go  back  and  decide,  for  each  one,  whether  the  corresponding  (unexpected) 
behavior  was  ever  indicated  in  the  transcript.  If  so,  identify  the  region  of  the 
transcript  in  which  it  indicated.  Use  the  segment  markers  described  in  the 
general  instructions,  i.e,  (=,  (,  [,  ],  ),  =).  Label  each  marker  with  the 
Rhetorical  or  Prohibitive  identifier,  (e.g,  R5  or  P5).  If  there  are  substantial 
gaps  (a  sentence  or  more)  in  the  region  in  which  the  response  is  supplied,  use 
several  sets  of  marked  brackets  rather  than  one  set. 

Final,  general  instructions. 

If  you  observe  any  place  where  one  participant  seems  clearly  to 
misunderstand  his  partner’s  Request,  underline  the  passages  which  led  you  to 
conclude  this  and  summarize,  in  your  own  words,  the  nature  of  the 
misunderstanding. 

If  you  find  a Request  that  seems  to  you  to  be  a repetition,  perhaps  with 
additional  detail,  of  a previous  question  posed  by  the  same  speaker,  indicate 
this  by  noting,  next  to  the  label  for  the  new  Request,  "="  followed  by  the  old 
Request's  label  (e.g,  "=Q5"). 

You  should  be  particularly  careful  with  utterances  which  appear  to  be 
Requests  but  which,  in  context,  are  only  presented  as  descriptions  of  behavior 
rather  than  a Request  to  perform  the  behavior.  For  example: 

A:  How  can  I see  better  in  here? 

B:  Turn  on  your  light,  stupid! 

A:  Oh,  thanks. 

The  point  of  this  example  is  that  B is  conveying  to  A the  information  he 
requested,  not  necessarily  asking  A to  do  anything  at  all.  For  these  cases,  the 
appa  ent  "Request"  (by  B)  is  simply  to  be  ignored. 

Figure  7-1  indicates  a segment  of  dialogue  annotated  according  to  these 
directions. 
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(<%L)  26  U:  OK  [operator’s  name]^CAN  YOU  PEOPLE  GO  INTO  ANY  DIRECTORY  AND  DELETE  FILES  WHEN  YOU 

27  NOTICE  THAT  THEY  ARE  NOT  BEING  USED^  [user’s  name] 


28  AM  REALLY  NOT  SURE,  HOWEVER  I KNOW  THAT  THAT  WOULD  NOT  BE 

29  RESPONSIBILITY  WE  WOULD  TAKE  UPON  OURSELVES  TO  JUDGE  WHETHER  OR 

30  NOT  YOU  WANT  YOUR  FILES  OR  NOT.^^ 


31  [outside  interruption  here] 


«SJO  32  0:  [user’s  namej^ARE  YOU  THERE?^ 


33  LINK  FROM  [user’s  name],  JOB  21,  TTY  22 


34  U:  SORRY  [operator’s  name] 


{Oi) 


WE  GOT  DISCONNECTED^CAN  YOU  RECOVER  THOSE  FILES  rQR  ME..^ 


AS  FAR  AS  I KNOW  THEY  WERE  IN  THE  DIRECTORY  ON  THE  16TH...THE  NAMES 


ARE  ...[file  name  1].DAT;1ABDLLLL  CHECK  THAT  ...[file  name  1].DAT;  1,2 
[file  name2].F4;l....AND  [file  name3].F4;l [user’s  name] 


C PI) 


(O  39  0^>^0LD  ON  JUST  A MINUT^AND  I WILL  TRY  TO  FIND  THEM)^ 


40  U:  RIGHT 


41  [outside  interruption  here] 


I 


Figure  7- la  - Dialogue  Annotated  for  Requests  - Part  a 


L&3)  42  O^ARE  YOU  STILL  THERE?^ 


IG3  4-)  43 

CQ.V)  44 
45 

(M+)  46 

47 

48 

(,0.6")  49 

(&2T  50 

51 

52 

COi)  53 

54 

55 


U^?IGHT^)perator’s  name] 

0:  OK  YES  I HAVE  FOUND  THE  FILES  YOU  ARE  CONCERNED  ABOUT^)N  THAT 
FIRST  ONE  I ASSUME  THE  ABDLLLL  WAS  AN  ERROR  RIGHT?^ 

l^RIGHT^HAT  SHOULD  HAVE  BEEN  [file  namel].DAT;l  AND  2. 

0:  OK  THEY  ARE  HERE 

[outside  interruption  here] 

O^ARE  YOU  THERE?  ^ 

l^<RIGHT>^ 

0:  OK  I HAVE  FOUND  THE  FILES  YOU  WANT 
[outside  interruption  here] 

0:  OK  I HAVE  FOUND  THE  FILES  YOU  WANT^WILL  RETRIEVE  THOSE  FOR 
YOU  ALSO^OK 

U:  GREAT. 


Figure  7-lb 


Dialogue  Annotated  for  Requests  - Part  b 


i 
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8.  EXPRESSION  OF  COMPREHENSION 

In  most  dialogues,  each  participant  wants  the  other  to  comprehend  what  he  says. 
Since  each  person  knows  this,  the  participants  often  say  things  that  serve  to  inform  the 
other  that  some  part  of  what  was  just  said  was  comprehended,  or  not  understood,  or 
partially  understood.  Sometimes  there  are  parts  of  the  dialogue  that  perform  only  this 
function,  and  sometimes  these  parts  do  this  while  conveying  other  content.  These  are 
all  aspects  of  Expressions  of  Comprehension  that  we  want  Observers  to  comment  on. 

There  are  two  components  of  this  mechanism  for  expressing 
Comprehension;  there  is  the  Expression  itself,  and  there  is  the  part  of  the  dialogue 

being  referred  to.  In  many  cases,  this  second  part  is  the  other  person’s  previous  turn. 
We  want  the  Observer  to  comment  on  both  of  these  parts. 

We  have  found  that  these  Expressions  occur  frequently  in  two  kinds  of  situations: 
those  with  noisy  channels  and  those  where  there  is  a high  cost  for  misunderstanding. 
We  expe:t  to  find  them  whenever  their  cost  to  the  speaker  is  lower  than  the  cost  of  the 
expected  reduction  in  misunderstanding. 

Here  are  the  instructions  for  the  Observer. 

OBSERVER'S  INSTRUCTIONS  FOR  EXPRESSION  OF  COMPREHENSION 

We  are  interested  in  the  Expression  by  a speaker  that  he  comprehends 
previous  dialogue.  There  are  various  ways  in  which  this  Expression  may 
occur: 


Three  Explicit  Examples: 


"OK,  That’s  clear." 


"Sure. 


M 


A:  "Is  that  clear?" 

B:  "Yeah." 

An  Implicit  Example: 

A:  "...so  that  proves  the  lemma." 

B:  "Using  that  same  method,  I can  prove  the  second 

one." 

Our  idea  of  comprehension  here  covers  both  hearing  what  is  said  and 
understanding  it.  We  are  interested  in  the  hearer’s  Expression  of  his  view  of 
his  own  comprehension  --  the  confidence  dimension  rather  than  the 
correctness  dimension.  (Thus  these  instructions  will  not  serve  to  identify 
cases  in  which  the  hearer  misconstrues  but  does  not  know  that  he 
misconstrues.) 

People  in  dialogue  indicate  various  degrees  or  scopes  of  comprehension  of 
what  the  other  party  has  said.  Using  the  directions  below,  we  want  you  to 
identify  places  where  someone  is  indicating  his  state  of  comprehension  of  what 
has  gone  before,  and  to  note  the  degree  and  scope  of  that  comprehension. 

Three  kinds  of  indication  of  comprehension  will  be  noted:  Positive 
Comprehension,  Noncomprehension,  and  Selective  Comprehension. 


Positive  Comprehension 

Scan  the  transcript  for  explicit  and  implicit  expressions  of  positive 
comprehension.  Mark  each  one  with  angle  brackets  , < and  >,  and  in 

parentheses  the  letters  PC  (for  "positive  comprehension")  and  a unique 
number,  e.g.,  (PC5). 

To  qualify  as  an  expression  of  positive  comprehension,  a region  must 
either  explicitly  express  comprehension  or  implicitly  indicate  this  by  specific 
dependence  on  what  was  said  before.  The  lack  of  an  Expression  of 
noncomprehension  is  not  sufficient  evidence  for  implicit  positive 
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comprehension.  Instead,  an  utterance  must  rely  on  some  specific  knowledge 
expressed  previously  by  (he  other  person. 


Some  of  these  Expressions  will  not  refer  specifically  to  any  definite  part 
of  the  dialogue. 

Others  will  clearly  refer  to  some  particular  previous  part.  For  the  latter, 
mark  the  part(s)  with  the  segment  markers  described  in  the  general 
instructions  , e.g.  <=,  (,  [,  ],  ) and  =).  Mark  each  of  these  markers  with  the  PC 
number,  e.g.  (PCS)  wnich  it  is  indicating  the  scope  of.  If  the  scope  of  the 
indication  is  exactly  the  previous  comment  by  the  other  speaker,  then  a slash 
with  the  identifying  number  may  be  used  instead  of  the  scope  brackets  to 
indicate  this;  e.g.  (PCS)/  . 

Multiple  indications:  Often  people  will  indicate  comprehension  in  more  than 
one  way  (according  to  our  notation).  For  example,  they  will  commonly  give  an 
explicit  indication  such  as  "Got  that."  followed  by  an  implicit  indication  such  as 
use  of  the  comprehended  material. 

RULE:  Annotate  the  earliest  indication.  The  scope  of  the  indication  should 
be  the  largest  contiguous  scope  which  is  either  all  Primary  or  all 
Nonprimary  according  to  the  directions  given  below. 

Use  this  rule  for  all  the  comprehension  categories. 

The  comprehension  may  be  expressed  to  various  degrees: 

PI.  Indefinite  degree:  Expressions  like  "Keep  going"  and  "I  guess  so" 
indicate  that  some  comprehension  has  occurred. 

P2.  Satisfactory  or  Substantially  Complete  Comprehension: 
Expressions  like  "OK"  and  "Sure"  sometimes  indicate  this  more  complete 
level.  Direct  repetition  without  significant  change  of  meaning  is  also  used 
to  indicate  this  level  of  comprehension.  (But  sometimes  it  indicates 
selective  comprehension  instead.)  Expression  may  include  approval  or 
agreement  or  consent. 


Mark  each  Expression  of  Comprehensiot.  with  an  indication  of  the  degree 
of  Comprehension  expressed.  Use  one  of  the  P numbers  above,  or  assign  a 
new  P number  of  your  own,  and  supply  ? descriptive  phrase  which  tells 
qualitatively  the  degree  of  comprehension  that  was  expressed.  If  you  use 
your  own  phrase,  try  to  express  its  rank  relati  ve  to  the  phrases  above,  e.g., 
more  than  PI  but  less  than  P2 
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Noncom  prehension 

Next,  scan  the  transcript  for  explicit  or  implicit  indications  of 
noncomprehension.  Phrases  like  "Huh?"  and  "Say  that  again"  may  indicate 
noncomprehension.  Mark  each  such  indication  with  NC  (for 
"noncomprehension")  and  a unique  number,  as  above,  and  indicate  the  scope  of 
the  noncomprehended  region,  if  known,  and  its  partialness,  if  it  is  clearly 
partially  noncomprehended.  Mark  them  in  a manner  directly  analogous  to  the 
above  directions  for  positive  comprehension. 

Expressing  a doubt  about  one’s  own  comprehension  is  a variety  of 
expressing  noncomprehension.  Even  when  in  fact  all  was  well  comprehended, 
an  Expression  of  doubt  should  be  marked  as  indicating  noncomprehension. 

Noncomprehension  may  be  expressed  to  various  degrees: 

Nl.  Indefinite  degree  of  noncomprehension. 

N2.  Substantially  complete  noncomprehension. 


Selective  Comi>rehcitsion 

People  sometimes  indicate  what  they  have  or  have  not  comprehended. 
Where  they  u so,  mark  the  region  with  angle  brackets  and  a unique  SPC 
number  or  SNC  number  (for  "Selective  Positive  Comprehension"  and  "Selective 
Noncomprehension"  respectively)  , as  is  done  for  PC  numbers  above.  Use  a 
new  SPC  or  SNC  number  for  each  selected  item.  To  qualify  as  being  selective 
comprehension,  there  must  be  some  indication  tha  there  are  differences  in 
how  well  the  various  candidates  for  comprehension  have  beer,  received. 

Indicate  the  region  for  which  the  comprehension  or  noncomprehension  is 
being  indicated,  as  above. 

If  you  identify  selective  positive  comprehension  within  a region  which  you 
also  mark  as  having  noncomprehension,  this  will  be  interpreted  as 
noncomprehension  of  all  except  the  selected  item(s).  Similarly,  you  can  show 
comprehension  of  all  except  selected  items  with  a PC  and  an  SNC.  for  each 
particular  thing  identified  selectively  as  comprehended. 

Use  the  phrase  marks  (such  as  PI  and  N2)  from  the  Positive 
Comprehension  and  Noncomprehension  directions  to  indicate  categories  of  SPC 
and  SNC. 
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The  Primary/Nonprimary  Distinction 

For  each  indication  of  comprehension  that  you  identified,  decide  whether  it 
is  used  by  the  hearer  primarily  to  indicate  his  comprehension,  or 
(alternatively)  whether  indication  of  comprehension  was  performed  along  with 
some  other  primary  function.  Mark  the  segment  identifying  number  with  ++  if 
indication  of  comprehension  is  the  primary  function  of  the  segment,  and 
with  if  it  is  not.  So  for  example  "huh?"  might  be  marked  as  (NC5)++, 
showing  that  it  was  identified  as  ?r>  indication  of  noncomprehension,  and  that 
the  primary  function  of  the  segment  was  to  indicate  the  speaker’s  lack  of 
comprehension. 


Use  of  the  material  comprehended,  such  as  answering  a question,  is  a 
common  nonprimary  indication  of  comprehension.  Simple  Expression  of 
comprehension  with  approval  or  agreement  or  consent  should  be  treated  as  a 
primary  indication  of  comprehension. 


( 
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People  sometimes  indicate  their  state  of  comprehension  by  telling  the 
other  person  what  to  do  about  that  state.  Expressions  like  "Keep  going"  and 
"Say  that  again?"  are  used  in  this  way.  A speaker’s  indication  of  his  state  of 
comprehension  in  any  of  these  ways  should  be  marked  with  ++. 


All  of  these  indicators  of  comprehension  deal  with  a person  expressing 
something  about  his  own  state.  Do  not  use  this  notation  for  cases  in  which 
one  person  is  indicating  that  the  other  one  is  confused,  has  failed  to 
comprehend,  has  comprehended  well,  or  something  similar. 


Notation  summary: 


identify  place  where  comprehension  was  expressed 
identify  place  which  Expression  refers  to 
segment  expressing  Positive  Comprehension 
segment  expressing  Noncomprehension 
segment  expressing  Selective  Positive  Comprehension 
segment  expressing  Selective  Noncomprehension 
expression  of  comprehension  is  primary  function 
expression  of  comprehension  is  not  primary  function 
(abbreviation)  expresses  comprehension  of  the  previous 
comment  by  the  other  speaker 


Figure  8-1  shows  an  example  of  applying  these  instructions  to  a sample  transcript. 
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c Pti) 

26  U:  OK  [operator's  name]. /c  AN  YOU  PEOPLE  GO  INTO  ANY  DIRECTORY  AND  DELETE  FILES  WHEN  YOU 

27  NOTICE  THAT  THEY  ARE  NOT  BEING  USED*J  [user’s  name] 

(Pci  ) 

28  O^l  AM  REALLY  NOT  SURE^HOWEVER  I KNOW  THAT  THAT  WOULD  NOT  BE 

29  RESPONSIBILITY  WE  WOULD  TAKE  UPON  OURSELVES  TO  JUDGE  WHETHER  OR 

30  NOT  YOU  WANT  YOUR  FILES  OR  NOT. 

31  [outside  interruption  here] 

32  0:  [user’s  name],  ARE  YOU  THERE? 

33  LINK  FROM  [user’s  name],  JOB  21,  TTY  22 

(AC  2.  )//**+•*- 

34  U^SORRY  [operator’s  name] 

35  WE  GOT  DISCONNECTED^CAN  YOU  RECOVER  THOSE  FILES  FOR  ME.. 

36  AS  FAR  AS  I KNOW  THEY  WERE  IN  THE  DIRECTORY  ON  THE  16TH...THE  NAMES 

37  ARE  ...[file  name  1 ].DAT;1  ABDLLLL  CHECK  THAT  ...[file  namel].DAT;1.2 

38  [file  name2].F4;l....AND  [file  name3].F4;l [user’s  name] 

(PC23/fJlH- 

39  0<OK)HOLD  ON  JUST  A MINUTE  AND  I WILL  TRY  TO  FIND  THEM 


I 

A 


Figure  8-1 


Dialogue  Annotated  for  Expression  of  Comprehension 


9.  SIMILAR  EXPRESSIONS  - TWO  KINDS 


OVERVIEW 

In  modelirg  communication  we  particularly  need  means  for  judging  how  well  the 
models  we  build  perform  comprehension  or  understanding.  Clues  about  what  the 
understanding  of  dialogue  depends  on  are  also  vital.  Developing  data  on  which  those 
judgments  ccn  be  made  is  an  essential  step  toward  making  evaluations  of  comprehension 
operational. 

The  Similar  Expressions  category  is  intended  to  provide  a basis  for  knowing 
whether  a model  is  extracting  too  much  or  too  little  or  the  wrong  things  from  particular 
items  in  a dialogue  transcript.  It  is  also  intended  to  provide  very  selective  information 
about  the  dependencies  of  interpretation  that  arise  between  parts  of  an  ongoing 
dialogue. 

We  are  exploring  this  with  two  similar,  but  independent,  experiments.  The  first  of 
these  calls  for  the  Observer  to  generate  alternative  expressions  which  he  feels  would 
accomplish  the  same  function  as  existing  expressions  from  a real  dialogue.  In  this  case, 
he  is  given  the  full  context  of  the  surrounding  dialogue.  The  second  approach  is 
significantly  more  complicated,  involving  the  generation  of  these  alternate  expressions, 
but  this  time  without  the  surrounding  dialogue.  The  Observer  is  then  given  the 
surrounding  dialogue  and  asked  to  separate  the  proposed  alternatives  into  those  which 
will  work  in  the  dialogue  and  those  which  will  not. 
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The  details  of  these  two  experiments  will  be  spelled  out  separately  below. 
However,  the  first  step  in  each  is  the  common  one  of  segmenting  the  given  dialogue  into 
"units."  This  will  be  detailed  first,  with  the  understanding  that  it  is  to  be  taken  as  the 
initial  step  in  each  of  the  processes  described  subsequently. 


Instructions  to  Unit  Divider 

You  will  be  given  a dialogue.  Divide  each  turn  into  "units,"  where  a unit  is 
either  a simple  sentence,  something  that  functions  like  a simple  sentence,  or 
the  whole  turn.  Do  not  feel  bound  to  observe  the  punctuation  you  find  in  the 
transcript  it  is  likely  to  be  inaccurate.  The  unit  boundaries  should  reflect  a 
single,  coherent  communication  having  approximately  the  "completeness"  of  a 
simple  English  sentence.  In  no  case,  however,  is  a unit  to  be  larger  than  a 
whole  turn.  Assign  a unique  number  to  each  unit. 


Similar  Expressions  In  Context 

We  would  like  to  compare  different  ways  of  saying  something,  i.e.,  ways  that  are 
reasonably  construed  as  being  equivalent  in  communication  effect.  We  will  describe 
such  expressions  as  Similar  and  intend  the  term  to  have  a specific  technical  meaning.* 
By  comparing  the  effects  on  the  model  of  a dialogue  item  with  the  corresponding  effects 
of  the  Similar  Expressions,  we  can  see  whether  the  models  are  creating  unnecessary 
differences  or  missing  important  common  effects. 


OBSERVER'S  INSTRUCTIONS  TOR  SIMILAR 


EXPRESSIONS  IN  CONTEXT 


We  are  interested  in  identifying  different  expressions  that  would  seem  to 
have  the  same  effect  in  a specific  dialogue. 

You  will  receive  a transcript  of  the  dialogue  that  has  been  divided  into 
"units"  (according  to  the  directions,  above,  for  Unit  Dividers).  For  each  unit, 
compose  one  or  more  Similar  Expressions  of  each  unit,  consistent  with  the 
following  constraint: 
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If  this  new  expression  had  occurred  instead  of  the  original  unit,  it 
would  have  been  acceptable  to  the  speaker  for  the  purposes  he  had 
in  mind.  You  should  base  your  judgment  only  on  the  dialogue  which 
precedes  the  utterance  in  question,  not  on  any  dialogue  which 
follows  it. 

We  encourage  you  to  use  different  words  and  styles  of  expression  than 
appear  in  the  given  units.  These  Similar  Expressions  constitute  the  results 
from  the  Observer,  for  this  experiment. 

EXAMPLE  OF  SIMILAR  EXPRESSION  IN  CONTEXT 


STANDARD: 

Can  you  recover  those  files  for  me? 


SIMILAR  EXPRESSIONS  IN  CONTEXT: 
Is  there  any  way  to  get  my  files  back? 
Please  restore  these  files  if  possible. 


♦Two  expressions  are  Similar  if,  in  your  opinion,  the  speaker  would  be 
willing  to  use  the  second  at  the  point  where  he  actually  used  the  first. 

We  attempt  to  express  this  "definition"  a little  more  clearly  with  the 
following  hypothetical  situation:  Imagine  a dialogue  between  a speaker  and  a 
hearer  who  did  not  share  a common  language,  but  who  had  translators 
available.  Imagine  further  that  in  order  to  insure  accuracy  of  communication, 
the  speaker  speaks  to  Translatorl  (only)  who  translates  what  he  hears  and 
communicates  this  translation  to  a second  translator  who  cannot  hear  the 
original  utterance.  The  second  translator  then  retranslates  the  utterance 
from  Translatorl  back  into  the  first  language  for  the  benefit  of  the  speaker. 
Thus,  the  speaker  is  afforded  the  opportunity  to  hear  his  own  words 
repeated  back  to  him,  after  undergoing  two  passes  of  translation.  At  this 
point  he  has  the  option  of  accepting  the  echoed  utterance  as  adequate,  or 
rephrasing  it  and  trying  again. 

Within  the  framework  of  this  hypothe'ical  situation,  two  expressions  are 
to  be  considered  Similar  if,  after  saying  the  first,  and  subsequently  hearing 
the  second  echoed  through  the  second  translator,  the  original  speaker  would 
be  willing  to  permit  the  differences  to  go  unchallenged  --  that  is,  he  accepts 
the  different  version  as  sufficiently  adequate  for  his  original  purposes  that  it 
does  not  need  revision. 
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Similar  Expressions  Out  Of  Context 

There  are  many  different  sources  of  information  that  the  interpretation  of  dialogue 
text  typically  depends  on.  They  include 


Situational  Information-  including  the  circumstances  of  the  dialogue, 
the  identities  and  cultural  roles  of  the  participants; 

Dialect  and  Jargon  Information-  The  local  conventions  of  language 
usage  that  were  in  effect  at  the  beginning  of  the  transcript. 

Context  Information-  The  part  of  the  transcript  that  precedes  the 
element  being  interpreted. 

Interpretation  depends  on  each  of  these  in  the  sense  that  for  a given  item 
(sentence,  word,  or  other)  variation  in  each  of  these  related  information  sources  can 
produce  variation  in  the  interpretation  of  that  item  in  dialogue.  (Notice  that  the  term 
Context"  is  being  used  in  a specific,  narrow  and  examinable  way.  It  is  not  a catchall 
term  for  "other  effects.") 

Because  we  are  dealing  explicitly  with  the  effects  of  each  item  in  the  transcript, 
identifying  effects  of  context  is  particularly  important.  These  directions  are  designed 
to  identify  these  effects  and  discriminate  them  from  the  others. 

The  basic  scheme  for  identifying  context  effects  is  sketched  in  Figure  9-1. 
Instructions  for  each  step  appear  below.  The  point  of  the  process  is  to  generate 
expressions  that  are  judged  Similar  in  the  out-of-context  equivalence  judgment,  and 
then  to  judge  the  acceptability  of  these  with  the  in-context  equivalence  judgment. 
THose  that  are  unacceptable  in  this  second  stage  must  be  so  because  of  some  effect  of 
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context.  We  ask  the  Observer  in  the  second  judgment  to  describe  factors  that  make 
such  items  unacceptable.  We  can  then  evaluate  a model  by  examining  whether  a 
corresponding  factor  (difference  of  effect)  arises  when  the  proposed  Similar  Expression, 
rather  than  the  original  item,  is  read  by  the  modei  at  the  appropriate  moment. 

011$ HR VEIi'S  INSTRUCTIONS  TOR  SIMII./IR  EXPRESSIONS  OUT  OF  CONTEXT 

We  are  interested  in  the  effects  of  the  preceding  dialogue  on  the 
interpretation  of  a turn  in  that  dialogue.  In  order  to  study  these  effects,  we 
have  devised  a multistep  procedure  which  makes  use  of  your  observations.  It 
starts  with  a dialogue  transcript,  isolates  pieces  of  some  turns  from  their 
surrounding  dialogue,  identifies  alternate,  Similar  Expressions,  not  knowing  the 
context  in  which  the  unit  originally  occurred,  and  finally  implants  these 
proposed  alternates  in  the  original  dialogue  at  the  point  where  the  original 
piece  had  been,  for  judgment  on  how  well  or  poorly  these  interpretations  fit. 

The  Similar  Expressions  are  prepared  out  of  context  (i.e.,  without  knowing 
what  preceded  or  followed  the  original  unit).  The  ones  that  turn  out  later  to 
fit  back  into  the  context  are  used  to  develop  methods  of  interpreting  concepts 
that  can  be  expressed  in  more  than  one  way.  The  Similar  Expressions  that 
are  later  judged  not  to  fit  are  used  to  identify  kinds  of  differences  between 
various  ways  of  saying  things,  leading  to  development  of  methods  for  being 
responsive  to  those  differences.  Both  the  Similar  Expressions  that  fit  into  the 
context  and  those  that  fail  are  valuable. 

(Several  Observers  will  perform  the  unit  division  task  on  different 
dialogues.  For  each  Observer  in  the  next  stage,  we  will  select  some  units 
from  each  of  these  dialogues  and  combine  them  into  a set  having  no  particular 
order  or  relation  between  the  units  as  they  are  presented.) 

SECI  ION  I:  I list  ructions  to  Generator  of  Similar  Expressions 

We  are  interested  in  different  ways  of  saying  something  which  have  the 
same  potential  effect.  We  want  you  to  produce  alternatives  to  some 
particular  units  that  have  occurred  in  actual  dialogue. 

You  will  receive  a set  of  units  For  each  of  these,  we  want  you  to  write 
one  or  more  new  units  (Similar  Expressions)  that  are  Similar  to  the  original 
unit  in  the  following  way:  In  your  judgment,  the  new  expressions  would  be 
regarded  by  the  speaker  as  having  about  the  same  effect  as  his  original  unit  in 
some  common,  ordinary  circumstance. 
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We  encourage  you  to  use  different  words  and  styles  of  expression  than 
appear  in  fhe  original  units.  Only  the  effect  which  you  think  the  speaker 
would  expect  needs  to  remain  the  same. 

Generating  these  Similar  Expressions  might  be  easier  if  you  proceed  in  the 
following  steps: 


1.  Read  the  unit. 

2.  Imagine  a situation  in  which  the  unit  could  occur. 

3.  Imagine  the  effect  intended  by  the  speaker. 

4.  Invent  a different  unit  which  would  have  the  same  effect  in  the 
situation  which  you  imagined.  Write  this  new  unit  down. 

5.  Repeat  steps  2 to  4 for  as  many  different  situations  as  come 
readily  to  mind.  (Note:  "different"  is  to  be  interpreted  as 
meaning  situations  which  would  suggest  a different  set  of 
admissible  Similar  Expressions.) 


You  should  try  to  imagine  ordinary  situations  rather  than  outlandish  ones, 
since  the  expressions  which  you  create  will  be  judged  by  other  Observers  on 
the  basis  of  whether  they  would  have  the  same  effect  in  some  ordinary 
circumstances. 

For  example,  in  attempting  to  generate  Similar  Expressions  for  the  unit: 
"yes,"  the  following  are  two  common  contexts  and  one  we  regard  as 
outlandish,  each  followed  by  a set  of  Similar  Expressions  appropriate  to  that 
context. 

1.  ("Is  he  your  boss?")  He  sure  is./That  he  is./He’s  the  one. 

2.  ( Do  you  want  to  go  to  the  movie?1')  You  bet. /Any  time  you’re 
ready./fine. 

3.  ("What’s  a three  letter  word  meaning  affirmative?")  Oui. 


(For  each  unit,  we  will  assemble  a list  of  all  Similar  Expressions  generated 
in  all  contexts  by  all  Observers.  The  next  step  Observers  will  be  given 
several  sets  of  these  expressions,  each  with  the  original  i lit  tnat  inspired  that 
set.  He  still  will  not  know  the  context  in  which  the  original  unit  occurred.) 
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SECTION  2:  Instructions  to  jud^e  of  Similar  Expressions  ( out  of  context). 

We  are  interested  in  whether  various  ways  of  saying  something  are  likely 
to  have  the  same  effect. 

You  will  receive  a collection  of  units,  divided  into  groups.  One  in  each 
group  will  be  called  the  Standard  Unit,  and  the  others  will  be  called  the 
Comparison  Units.  You  are  to  judge  each  Comparison  Unit  according  to  the 
following  criterion:  In  your  judgment,  would  the  effect  of  the  Comparison  Unit 
be  acceptable  to  the  speaker  of  the  Standard  Unit  in  some  ordinary 
circumstances? 

Mark  each  Comparison  Unit  with  one  of  the  symbols:  "+",  or  V for 
Acceptable,  Unacceptable,  or  Unclear. 

(The  Observer  for  the  final  step  will  be  given  one  of  the  original 
transcripts  w*'.n  the  units  numbered.  For  each  numbered  unit,  he  will  also  be 
provided  with  a set  of  those  Similar  Expressions  for  that  unit  which  received  a 
"+"  in  the  prior  step’s  grading.) 


SECTION  3:  Instructions  to  jud/rc  of  Similar  Expressions  (in  context). 

We  are  interested  in  how  the  effect  of  what  is  said  is  related  to  the 
context  in  which  it  is  said.  We  want  you  to  make  some  judgments  about 
whether  or  not  certain  Similar  Expressions  for  a unit  can  actually  take  the 
place  of  that  unit  in  context,  and,  if  not,  why  not. 

Materials: 

You  will  receive  a transcript  of  an  actual  dialogue,  as  it  occurred,  divided 
into  numbered  units.  For  each  numbered  unit,  you  will  also  receive  a set  of 
possible  alternative  expressions  for  it. 

Commentary: 

For  each  one  of  the  Similar  Expressions  given  to  you,  make  two  kinds  of 
annotation: 

1.  If  this  expression  had  occurred  instead  of  the  original,  would  it  have 
been  acceptable  to  the  speaker  of  the  original?  Mark  it  with  either  or 

for  Yes,  No,  or  Unclear.  You  should  project  the  acceptability  of  the  turn 
based  only  on  the  part  of  the  dialogue  that  precedes  the  unit,  not  on  anything 
which  follows. 
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2.  If  you  answer  then  describe  very  briefly  one  important  difference 
between  the  original  and  the  alleged  Similar  Expression,  from  the  point  of  view 
of  the  speaker.  If  the  expression  is  incomprehensible  in  this  context,  mark  it 
with  "X”,  meaning  Incomprehensible  (even  if  it  might  be  meaningful  in  other 
contexts). 

These  annotations  constitute  the  results  of  the  second  experiment. 

EXAMPLE  OF  SIMILAR  EXPRESSIONS  GENERATED  OUT  OF  CONTEXT 

STANDARD  UNIT: 

As  far  as  I know  they  were  in  the  directory  on  the  16th. 


RESULTS  FROM  SECTION  1: 


I.  "Ttay  tvtr-ewV  qftii  lot-fer*. 
X.  J 1 link  *Hwfr  pUov*  LuQS 

i;M  ^ tie  fltL 

3 fry  f'lei  Ml  tie*** 

"H\e  ILtk . 

y.  Tt ft  he**  bee*  n* 
"tie  Jlnr/e-Zory  tie  Ifc'Hv 

RESULTS  FROM  SECTION  2: 

(.  +• 
t.  — 

3.  + 

y.  f 


CONTEXT  PRECEDING  STANDARD  UNIT: 

Can  you  recover  those  files  for  me? 

RESULTS  FROM  SECTION  3: 

I.  * 

3 . t 

v.  -(y\u  tvi  IdJUr  c,ltar(p  V 'e4W\r 

fe  Jmti,  Hot  of 


"f»>!  ) 
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10.  TOPIC  STRUCTURE 


The  participants  in  a conversation  often  continue  to  talk  about  a particular  set  oi 
concepts  for  several  turns.  This  related  set  of  ideas  constitutes  a Topic  of  discussion. 
Topics  may  proceed  for  a long  time,  as  in  a long  telephone  discussion  of  vacation  plans, 
or  may  proceed  for  only  two  short  turns,  as  in  a simple  question  about  the  time  its  the 
answer.  Topics  may  be  general  or  specific,  and  the  participants  may  be  discussing 
more  than  one  Topic  at  a time.  Topic  is  a unit  based  on  the  content  being  discussed 
rather  than  the  forms  being  used.  Some  parts  of  the  discussion  may  be  part  of  no 
clearly  distinct  Topic  - for  example,  the  negotiation  of  what  to  discuss  next.  The 
existence  and  nature  of  units  of  discourse  larger  than  a single  sentence  is  one  of  the 
important  issues  being  currently  pursued  by  a number  of  researchers.  The  Topic 
annotation  described  here  may  shed  light  on  the  currently  hypothesized  supersentential 
structures,  such  as  "frames"  (Minsky,  1974),  "scripts"  (Schank  & Abelson,  1975),  and 
"story  grammars"  (Rumelhart,  1975). 

Here  is  the  current  set  of  instructions  for  annotating  Topics  in  dialogue: 


0 IIS  PRV  PR'S  INSTRUCTIONS  TOR  CODING  TOPICS 

In  most  dialogues  there  are  one  or  more  Topics  which  are  being  discussed. 
A Topic  is  a subject  discussed  in  one  or  more  turns  of  the  dialogue. 

We  want  you  to  go  through  the  dialogue,  marking  those  segments  in  which 
a Topic  is  being  discussed.  Annotate  the  beginning  and  ends  of  these 
segments  with  the  Segment  Markers,  described  above.  Note  that  segments 
may  contain  other  embedded  segments. 
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Do  this  for  each  person,  separately  in  the  dialogue.  Also,  draw  a line  from 
the  initial  marker  for  each  Topic  to  the  margin  (left  for  speaker  Aj  right  for 
speaker  B)  and  write  a brief  descriptive  title  for  the  Topic.  Use  this  same 
title  whenever  the  same  Topic  reappears,  in  either  speaker’s  utterances. 
Notice  that  not  every  part  of  the  transcript  has  to  be  part  of  a Topic. 


When  you  have  finished  annotating  the  Topics  of  the  dialogue,  list  any 
Topics  st'll  open  at  the  end  of  the  transcript.  Also  list  any  Topics  that  were 
already  open  when  the  part  of  the  dialogue  you  have  annotated  started.  To 
check  these,  make  sure  that  every  Topic  marked  as  opened  is  either  marked  as 
being  closed  or  listed  as  still  open.  Do  the  same  for  every  annotation  of  Topic 
closing. 


Figure  10-1  is  an  example  of  an  annotation  resulting  from  the  application 
of  these  instructions  to  a piece  of  dialogue. 


53  0:  OK  I HAVE  FOUND  THE  FILES  YOU  WANT  I WILL  RETRIEVE  THOSE  FOR 


It  s 


54  YOU  ALSO,  OK 


55  U:  GREATi w/AS  THAT  [name3]  YOU  WERE  TALKING  ABOUT? 

YesoU  cd 



J 


r 1 1 vi*f\ 

*'Ut  J 


56  O/NO,  IT  IS  [name4],  HOWEVER  WHEN  YOU  SEND  MESSAGE 


VtfLt  ui 
C oMJvtir 
q€i ktur 


57  SEND  TO  [name2] 


58  U:  RIGHT...IS  [name5]  STILL  AR[outside  interruption  here] 


59  IS  [name5]  STILL  AROUND. 


60  0:  NO  HE  ISN’T.  HE’S  BEEN  GONE  FOR  ABOUT  THREE  WEEKS  NOW. 


PconLt  U:  O^THANKS  FOR  YOUR  HELP  , [operator’s  name]  HAVE  A GOOD  DAY  [user’s  name]  OUT 

iuV.p.tit  _J 

Ct*-1**-  i PtyUJiL 

62  0:  OKJYOU  DO  THE  SAME  WILL  DO  THOSE  FILES  RIGHT  AWAY.  BYE 


63  U:  THANKS  AGAIN 


64  BREAK 


1 tf't-S  dlrOfJy 

o>  4jL<| 


O ; UXr'itM/A/c^  f;lei 
^ - dtkr  t4.tr/wy  f)  (q  j 


cjf  e<Ajt 


K-«V\^ 


Figure  10-1  - Dialogue  Annotated  for  Topic 
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SHARED  TOPIC  POST-PROCESSING 

In  our  early  directions,  we  attempted  to  have  the  Observer  annotate  tlw  topic 
shared  by  the  two  participants.  This  led  us  into  considerable  difficulty,  since  the  topics 
addressed  by  the  individual  participants  were  frequently  "out  of  step"  with  each  other 
in  nontrivial  ways.  From  our  attempt  to  capture  the  notion  of  one  common  topic  of  a 
dialogue,  we  developed  the  concept  of  a particular  participant’s  topic.  Thus,  the  two 
participants  may  be  talking  about  different  topics  at  the  same  time,  as  often  occurs  at 
topic  boundaries,  when  one  starts  a new  topic  before  the  other  finishes  talking  about 
the  old  one.  This  notion  of  a individual’s  topic  is  imbedded  in  the  instructions  given 
above. 


However,  we  found  that  we  could  derive  from  these  observations  of  an  individual’s 
topic  an  annotation  of  shared  topic  (involving  no  additional  subjective  judgments), 
following  the  simple  algorithms  given  below. 


Post-Processing  Procedure  For  Shared  Topics 

We  will  use  the  Observer’s  judgments  of  topic  boundaries  to  determine  the  period 
during  which  the  two  participants  share  a topic  of  conversation. 


The  general  intention  embedded  in  the  algorithm  is  as  follows:  A participant  is 
sharing  the  topic  currently  being  discussed  by  his  partner  unless  the  Observer  says 
that  he  definitely  is  not. 
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The  detailed  blow-by-blow  procedure  is: 


1)  To  mark  the  beginning  of  a shared  topic: 


a.  Find  a shared  topic  (T)  (a  topic  discussed  by  both  participants  and  given  the 
same  name  by  the  Observer). 

b.  Find  the  place  that  each  definitely  starts  talking  about  T (the  inner  starts). 

c.  Call  the  earlier  start  SE;  call  I he  later  start  SL. 

d.  For  the  person  who  spoke  SL,  (PL),  find  the  earliest  point  in  the  speech  of 
either  party  at  which:  (SE  had  already  occurred)  and  (PI  had  not  spoken 
anything  which  occurred  prior  to  the  outer  start  of  topic  T by  PL). 

This  is  the  Topic  Sharing  Begin  (TSB). 

2)  To  mark  the  end  of  a shared  topic: 


a.  For  each  shared  topic  (T)  marked  as  beginning,  above. 

b.  Find  the  places  where  it  definitely  ends  for  each  participant  (the  inner  ends). 

c.  Call  the  later  end  EL;  call  the  earlier  end  EE. 

d.  For  the  person  who  spoke  EE,  (PE),  find  the  latest  point  at  which  (EL  had  not 
yet  occurred)  and  (PE  had  not  spoken  anything  which  occurred  after  the  outer 
end  of  topic  T by  PE). 

This  is  the  Topic  Sharing  End  (TSE). 

3)  If,  after  steps  1 and  2,  it  is  discovered  that  the  TSE  precedes  the  TSB,  then  merely 
delete  both  marks. 


4)  The  region  of  sharing  of  topic  T runs  from  the  TSB  to  the  TSE. 


■ 
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The  following  examples  are  intended  to  illustrate,  in  an  abstract  form,  the  desired 
intent  of  these  directions.  The  five  examples  represent  two  utterances  each  by  A and 
B,  alternating,  in  the  course  of  which  a shared  topic  is  initiated.  The  segment  markers 
within  the  sequences  of  A’s  and  B’s  indicate  their  position  within  the  dialogue  dictated 
by  the  Topic  Annotation  directions.  On  the  line  immediately  below  this,  the  dashed  line 
indicates  the  region  of  the  dialogue  for  which  the  topic  is  deemed  to  be  shared.  (On 


| 
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the  belief  that  start  and  end  instructions  are  symmetric,  only  starts  are  shown.) 


1)  fAAAAAAAAA  [J3BBB^3BBB  AAAAAAAAAA  BBBBBBBBBB 
> ... 


2)  [AAAAAAAAA  BBBBjkBBBB  AAAAAAAAAA  BBBBBBBBBB 

> . 


3)  AAAAAAAAAA  BBBBBB(sBB  AAAA£AAAAA  BB(BBBBBBB 

> 


4)  AAAAAAAAAA  BBB(sBB(BB  AAAAAAAAAA  BBBBBBBBBB 

> ... 


5)  AAAAAAAAAA  BBBBBBBBBB  AAAAAAAAAA  (*BB^3BBBB 

> ... 


In  figure  10-2,  we  exhibit  an  example  of  the  annotation  that  results  from  applying  this 
procedure  to  the  individual  topic  example  given  previously. 


53  0:  OK  I HAVE  FOUND  THE  FILES  YOU  WANT  I WILL  RETRIEVE  THOSE  FOR 


54  YOU  ALSO,  OKI 


Cl 


U:  GREAVIvAS  THAT  [name3]  YOU  WERE  TALKING  ABOL 


*6  0:[N0,  IT  IS  [name4],  HOWEVER  WHEN  YOU  SEND  MESSAGE 


" fofthwivi 

r~;  Li " 


57  SEND  TO  [name?] 


58  U:  RIGHT...IS  [name5]  STILL  ARfoutside  interruption  here] 

59  IS  [name5]  STILL  AROUND 


60  0:  NO  HE  ISN’T.  HE’S  BEEN  GONE  FOR  ABOUT  THREE  WEEKS  NOW. 
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II.  INCREMENTAL  AND  PREREQUISITE  INFORMATION 


One  of  the  most  difficult  aspects  of  a dialogue  to  grasp  (at  least  for  our  modeling 
efforts)  is  what  we  loosely  call  the  informational  content.  In  order  to  somewhat  confine 
our  Observers,  we  have  specifically  limited  their  annotations  on  the  information 
evidenced  by  the  speakers,  to  that  relevant  to  the  topics,  previously  annotated.  Since 
we  felt  any  choice  of  formal  notation  for  informational  content  would  seriously  bias  the 
Observers,  we  have  opted  for  simple  English  as  the  medium  of  expression  for  these 
observations. 

Finally,  we  have  asked  the  Observer  to  differentiate  two  kinds  of  information:  that 
conveyed  by  the  dialogue  and  that  which  was  never  expressed  but  must  have  been 
present  for  the  dialogue  to  be  understandable  to  the  participants. 

Although  we  are  in  full  agreement  concerning  the  importance  of  this  particular 
category,  we  have  been  unable  to  achieve  even  a partial  consensus  on  any  proposed 
set  of  directions.  We  remain  committed  to  the  attempt  to  deal  with  this  dimension  but 
do  not  believe  the  current  state  of  the  directions  merits  being  reported  here. 
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12.  CORRECTION  ACTIONS 


Every  so  often  in  the  course  of  a dialogue,  one  of  the  participants  becomes  aware 
of  a specific  bit  of  misinformation  which  Ins  partner  seems  to  be  operating  with.  When 
the  speaker  sets  about  to  repair  this  unfortunate  state  of  affairs,  we  say  he  is  engaged 
in  a Correction  Action.  Whenever  we  find  one  of  these  actions,  we  would  like  to 
identify: 


1)  Where  did  the  correction  Occur? 

2)  What  indicates  that  this  is  a correction7 

3)  What  was  being  corrected7 

4)  How  do  we  know  what  is  being  corrected? 

5)  How  is  the  correct  information  indicated? 

6)  How  could  the  correction  have  been  avoided? 

7)  Did  the  recipient  of  the  correction  suggest  it7 

8)  Was  the  corrector,  himself,  in  error7 


QilSERVRR'S  INSTRUCTIONS  TOR  CORRECTION  ACTIONS 


We  are  interested  in  how  people  make  corrections  during  the  progress  of 
a dialogue.  We  want  to  identify  corrections  that  occur,  and  eventually 
understand  how  they  are  accomplished  and  what  their  effects  are. 

For  the  purposes  of  this  set  of  observations,  we  want  to  focus  only  on 
particular  kinds  of  corrections,  ignoring  all  others.  The  Correction  Actions 
that  we  want  you  to  identify  and  annotate  must  have  the  following  two 
properties: 


A.  Retraction/Cancellation  Property: 

Something  in  the  previous  dialogue,  which  has  been  comprehended  in  some 
way  by  each  participant,  is  retracted  or  cancelled  by  the  further  utterances  of 
one  of  the  participants, 
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Things  retracted  occur  in  a variety  of  forms.  For  example,  the  retraction  may  be 
effected  by  talking  about  retraction  in  some  way,  or  it  may  occur  as  part  of  a 
substitution  or  revision. 

The  thing  corrected  and  the  correction  may  occur  in  the  same  turn;  in  this  case  it  is 
sufficient  to  find  that,  in  the  absence  of  the  correction,  the  recipient  would  probably 
have  comprehended  the  part  of  the  comment  being  corrected  in  another  way. 

The  Correction  Action  may  span  several  turns  by  the  person  doing  the  correcting, 
or  it  may  be  complete  in  a single  turn.. 

B.  Explicitness  Property: 

There  must  be  some  explicit  indication  that  a Correction  was  intended  (e.g.: 

"No,  I meant  ...  ",  "Sorry,  make  that  ...  ",  ...).  So  repetitions,  clarifications, 

explanatory  elaborations  and  restatements  are  not  usually  Correction  Actions. 

In  our  experience,  Corrections  having  these  two  properties  are  relatively  rare. 
Many  dialogues  do  not  contain  any.  On  the  other  hand,  it  is  possible  to  find  dialogues 
in  which  they  are  relatively  frequent. 

The  annotations  are  to  be  done  in  two  stages,  on  separate  copies  of  the  transcript. 
The  first  stage  deals  with  the  regions  which  do  the  correcting;  the  second,  with  the 


regions  being  corrected. 


ANNOTATIONS  FOR  CORRECTION  ACTIONS  - first  pass 
A.  CORRECTION  REGION  (CR) 


Before  you  can  annotate  any  of  the  details  of  a Correction,  you  must  first  ascertain 
that  a Correction  has  occurred.  When  you  find  a region  of  the  dialogue  which  satisfies 
the  two  requirements  above  (a  Correction  Region),  bracket  the  region  as  you  would  a 
topic  and  assign  the  region  a unique  label:  CR  followed  by  a number  (e.g.  CR7).  The 
Correction  Region  is  to  start  with  the  first  indication  that  a Correction  is  under  way 
(this  would  include  a Correction  Request"  by  the  recipient,  see  below)  and  end  when 
the  Correction  is  no  longer  under  discussion,  for  example, 

A:  ...  out  window  three,  [correction  on  that!  That's  out  window  one] 

' — (ce3) 

Beyond  this  point,  you  should  be  aware  that  not  all  of  the  requested  annotations 
will  apply  to  each  Correction.  Likewise,  as  usual,  it  may  not  always  be  obvious  which 
classifications  apply  or  where.  In  either  case,  you  are  simply  to  ignore  the  inapplicable 
instructions  or  those  whose  application  is  not  obvious. 

Underline  and  label  each  of  the  regions  described  below  with  their  two-letter  code 
(e.g.:  CP,  CC,  CQ,  ...)  followed  by  the  number  assigned  to  the  corresponding  Correction 
Region.  You  might  find  that  some  of  these  regions  overlap  or  are  even  identical  with 
others--that’s  perfectly  all  right. 
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8.  CORRECTION  FLAG  (CF) 

A consequence  of  the  Explicitness  Property  is  that  there  is  almost  always  a part  of 
the  Correction  Region  (called  the  Correction  Flag)  in  which  the  corrector  signals  that  a 
Cor.ection  is  taking  place,  (e.g.  "Oops,  what  I meant  was  ...  " , "Wrong-o!,  the  right 

answer  is  ...  , ...)  For  each  Correction  Region  you  bracket,  find  its  Correction  Flag,  for 
example, 

A:  ...  out  window  three.  ^Correction  on  that!  That’s  out  window  one.-} 

^ (C F8  ) 

C.  CORRECTION  POINTER  (CP) 

Wiihin  the  Correction  Region,  the  corrector  will  use  certain  words  to  indicate  what 
part  of  the  preceding  dialogue  he  wants  to  correct.  This  region  (the  Correction 
Pointer)  will  not  represent  any  new  or  different  information,  for  example, 

A:  ...  out  window  three.  ^Correction  on  that!  That’s  out  window  one.]] 

D.  CORRECTION  CONTENT  (CC) 

In  addition  to  locating  the  Error  Region  with  the  Correction  Pointer,  the  Correction 
Region  will  also  indicate  the  nature  of  the  substitution  or  revision.  This  novel 


’s  out  window  one. 


•) 


"Q 


C.C.1) 


information  is  the  Correction  Content,  for  example, 

A:  . out  window  three.  [Correction  on  that!  That 
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E.  CORRECTION  REQUEST  (CQ) 


Occasionally,  the  recipient  of  an  error  will  explicitly  indicate  his  doubts  that  his 
partner  indeed  meant  what  he  clearly  said.  (E.g.:  "Did  you  mean  ...  or  ...?",  "Shouldn’t 
that  be  ...  ?",  ...)  If  you  see  this  happening,  note  this  as  a Correction  Request.  Note 

that  this  will  then  be  the  beginning  of  a Correction  Region,  for  example, 

A:  ...  out  window  three. 

B:  £Are  you  sure  you  mean  three? 


A:  Correction  on  that!  That’s  out  window  onej 
Figure  12-1  gives  an  example  of  first-pass  Correction  Action  annotation. 


ANNOTATIONS  FOR  CORRECTION  ACTIONS  - second  pass 

On  the  second  copy  of  the  dialogue,  write  the  brackets  and  labels  from  the 
Correction  Regions,  as  annotated  in  the  first  pass.  Once  this  is  done,  proceed  with 
annotating  the  regions  described  below. 

A.  ERROR  REGION  (ER) 

In  most  cases,  the  corrector  will  be  making  his  Correction  as  a modification  of  some 
prior  region  of  the  dialogue  which  contained  the  error.  This  part  of  the  dialogue  (the 
Error  Region)  may  be  in  his  own  words,  or  his  partner’s.  For  each  Correction  Region, 
find  the  corresponding  Error  Region,  for  example, 

A:  ...  out  window  three.  Correction  on  that!  That’s  out  window  one!! 

1 


(e*  3> 
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34  U:  SORRY  [operator’s  name] 

35  WE  GOT  DISCONNECTED....CAN  YOU  RECOVER  THOSE  FILES  FOR  ME.. 

36  AS  FAR  AS  I KNOW  THEY  WERE  IN  THE  DIRECTORY  ON  THE  16TH...THE  NAMES 

37  ARE  ...[file  namel j.DAT;lABDLLLL  f CHECK  THAT  ...[file  name  1].DAT;  1,2*1 

38  [file  name2].F4;l....AND  [file  name3].F4;l....J[user’s  name] 


].F4;l...Juser’s 


39  0:  OK  HOLD  ON  JUST  A MINUTE  AND  I WILL  TRY  TO  FIND  THEM 


tr 


(tci) 
{CP}) 


40  U:  RIGHT 


41  [outside  interruption  here] 


42  0:  ARE  YOU  STILL  THERE? 


43  U:  RIGHT  [operator’s  name] 


44  0:  OK  YES  I HAVE  FOUND  THE  FILES  YOU  ARE  CONCERNED  ABOUt|qN  THAT 

45  FIRST  ONE  I ASSUME  THE  ABDLLLL  WAS  AN  ERROR  RIGHT? 


Kc.n.%) 


Uat) 


46  U:  RIGHT  THAT  SHOULD  HAVE  BEEN  [file  namel].DAT;l  AND  2.“| 

1 i T-Ucj.) 

47  0:  OK  THEY  ARE  HERE  P 4l\ 

(C  FI ) ’ 


Figure  12-1  - First-pass  Correction  Action  Annotations 
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The  error  region  should  include  not  only  the  text  which  is  actually  corrected,  but 
also  that  which  is  used  by  the  Correction  Pointer  (see  below)  to  locate  the  area  to  be 
corrected. 


Sometimes,  the  corrector  is  not  correcting  preceding  dialogue,  but  rather  an 
erroneous  concept  which  he  believes  the  recipient  holds.  If  you  see  this  happening, 
describe  this  erroneous  concept  in  the  margin  and  treat  it  as  the  Error  Region. 

B.  CORRECT  ALTERNATIVE  (CA) 


Frequently  it  is  easy  to  see  how  the  error  which  was  corrected  could  have  been 
avoided  in  the  first  place.  If  possible,  select  a region  of  the  dialogue  (usually  the  Error 
Region)  and  rewrite  it,  using  as  nearly  as  you  ran  the  style  of  the  speaker,  so  that,  had 
the  substitute  been  used,  the  net  effect  would  have  been  the  same  as  the  actual 
utterance  and  subsequent  Correction,  for  example, 

A:  ...  out  window  three.  ^Correction  on  that!  That’s  out  window  one^ 


(.C*3) 

0<JT  UifArbOU) 
OAf£ 


C.  CORRECTOR  ERROR  (CE) 


If  you  encounter  a situation  where,  in  your  judgment,  the  corrector  makes  a 


Correction  which,  if  successful,  will  have  no  net  effect,  indicate  this  with  a STAR  (*)  in 
the  margin,  alongside  the  Correction  Region,  for  example, 
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34 

35 

36 

37 

38 

39 

40 

41 

42 

43 

44 

45 

46 

47 


U:  SORRY  [operator’s  name] 

WE  GOT  DISCONNECTED....CAN  YOU  RECOVER  THOSE  FILES  FOR  ME.. 

AS  FAR  AS  I KNOW  THEY  WERE  IN  THE  DIRECTORY  ON  THE  16TH...THE  NAMES 
ARE  ...[file  namel].DAT;lABDLLLL  [CHECK  THAT  ...[file  namel], 


[file  nameE].F4;l....AND  [file  na 


.DAT;  1,2] 

cc  m 


)Jne3].F4;l [user’s  name] 

Cent),  (<r*>2) 

(C410  ? Lf.'le  .D/9-;l  l. 

0:  OK  HOLD  ON  JUST  A MINUTE  AND  I WILL  TRY  TO  FIND  THEM 


U:  RIGHT 


[outside  interruption  here] 


0:  ARE  YOU  STILL  THERE? 


U:  RIGHT  [operator’s  name] 

0:  OK  YES  I HAVE  FOUND  THE  FILES  YOU  ARE  CONCERNED  ABOUTfON  THAT 

FIRST  ONE  I ASSUME  THE  ABDLLLL  WAS  AN  ERROR  RIGHT?  V {CR.%) 

U:  RIGHT  THAT  SHOULD  HAVE  BEEN  [file  namel].DAT;l  AND  2^ 


0:  OK  THEY  ARE  HERE 


Figure  12-2  - Second-pass  Correction  Action  Annotations 
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13.  SUMMARY  AND  PLANS 


The  report  above  has  described  the  very  substantial  progress  that  has  been  made 
on  identifying  suitable  dialogue  phenomena  for  this  work.  The  observation  methods  are 
all  in  a state  of  incomplete  development,  which  will  necessarily  persist  for  some  time. 
Since  they  are  not  in  a finished  state,  it  is  not  timely  to  try  to  make  them  completely 
adequate  for  widespread  use  or  formally  assess  their  reliability. 

The  substantive  content  of  the  observational  categories  must  be  fitted  to  the 
modeling  process  in  order  to  have  a smoothly  operating  methodology.  This  fitting  has 
not  yet  been  done.  Until  it  is,  the  basic  definitions  of  the  categories  will  not  be  stable. 
Therefore  formal  validation  work  or  extensive  documentation  work  on  the  observational 
methods  is  not  appropriate  at  this  time.  The  impact  of  later  modeling  activity  would 
effectively  cancel  any  immediate  work  of  either  kind. 

On  the  other  hand,  there  is  value  in  exposing  the  methodology  and  observational 
categories  to  technical  and  personal  points  of  view  not  represented  among  the  authors. 

We  plan  to  construct  process  models  for  one  or  two  very  short  dialogues 
immediately.  The  purpose  of  these  models  will  be  to  sketch  the  form  of  the  processes 
needed  and  determine  major  relationships  between  parts.  This  stage  will  necessarily 
involve  a subset  of  the  Observers’  categories,  relatively  rough  representation  of  some 
kinds  of  knowledge,  and  short  dialogues. 
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As  our  modeling  skill  develops,  we  expect  to  deal  with  more  phenomena  per  model, 
with  longer  dialogues,  and  with  higher  fidelity  to  ihe  actual  communication  events. 

Past  experience  on  related  problems  indicates  that  such  models  often  converge 
rapidly  on  some  relatively  effective  processes,  each  episode  of  accounting  for  new  data 
taking  substantially  less  work  than  the  previous  one.  When  this  convergence  takes 
place,  it  is  indicative  of  success.  Also,  ability  to  model  rapidly  makes  it  possible  to  build 
successively  more  ambitious  models. 

Different  parts  of  these  models  will  mature  at  different  rates.  It  is  to  be  expected 
that  some  processes  will  be  ready  for  transfer  into  working  man-machine  systems 
relatively  soon,  and  others  much  later.  We  plan  to  identify  and  report  on  those  that 
appear  to  be  ready  for  transfer  on  a continuing  basis. 


.u Biipii igyj, lnlWKi  . .„  , „IIL.  . JI!L.,UI|,4JII.!l(llJjpj 
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APPENDIX 

FULL  TEXTS  OF  THE  DIALOGUE  OF  THE  EXAMPLE 

The  examples  used  in  the  report  are  not  character-by-character  reproductions  of 
the  original  typed  dialogue.  Some  differences  have  been  introduced  for  various 
reasons,  including  privacy,  ease  of  use,  and  a desire  to  focus  on  certain  phenomena. 
This  section  makes  it  possible  to  identify  some  of  the  differences  by  showing  a text 
which  more  strongly  resembles  the  sojre  text,  but  with  privacy  preserv-  ' und  genuine 
irrelevancies  deleted.  (The  text  which  resembles  the  source  is  shown,  first,  then  the 
cleaned-up  version  prepared  for  the  Observer.) 

We  have  chosen  to  use  cleaned-up  data  in  order  to  limit  the  diversity  of  phenomena 
which  need  to  be  dealt  with  at  once.  Because  the  differences  introduced  in  this  way 
seem  distinct  from  the  phenomena  of  central  interest,  is  therefore  a reasonable  tactic  to 
eliminate  them  in  the  immediate  future,  without  in  any  way  ruling  them  out  of  the  scope 
of  the  general  problem  of  understanding  how  communication  works. 

The  transcript  below  represents  communication  between  a computer  operator  and  a 
user  of  the  computer.  Their  terminals  have  been  linked  together  by  use  of  the  LINK 
command,  which  causes  each  character  which  appears  at  either  terminal  to  appear  at 
the  other  as  well.  The  transcript  differs  from  the  original  only  in  the  following  ways: 

1.  Material  deleted  is  noted  by  [square  brackets] 

2.  "0"  indicates  the  operator’s  comments,  and  "U"  the  user’s. 

3.  Each  turn  by  a speaker  has  been  set  off  by  a blank  line  and  these  labels. 
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The  spelling,  grammar,  punctuation,  etc.  are  those  that  originally 
occurred. 


8 LINK  FROM  [user’s  name],  JOB  21,  TTY  3 

9 U:  HELLO  ARE  YOU  THERE...GA 

10  0:  YES,  GO  AHEAD 

11  U:  A COUPLE  A QUESTIONS....ARE  THE  FILES  THAT  ARE  ON  DISK  ARCHIVED  EVERY 

12  WEEK/ 

13  0:  YES 

14  U:  I XXX 

15  IN  ONE  OF  MY  DIR.  I HAD  SOME  D/  FILES  IN  ONE  DAY  AND  THE  NEXT  THFY  WERE 

16  GONE  AND  THEY  WERE  NOT  ARCHIVED/ 

17  0:  WELL,  I AM  REALLY  NOT  THAT  FAMIL  SURE  OF  HOW  THE  ARCHIEING  OF 

18  THE  FILES  ARE  DONE  SO  I WOULD  SUGGEST  THAT  YOU  SEND  A MESS.  TO 

19  [name2]  IN  REQ\QGARDS  TO  THIS  PROBLEM.  THE  ARCHIE\EVEING  IS  DONE 

20  DURING  THE  SWING  SHIFT  SO  IN  ORDER  THAT  YOU  DO  DONT  NOT  GET 


1 


21 


MES  MISLED  BY  SOMETHING  I AM  NOT  CERTAIN  ON  I WOULD  RATHER 
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22  YOU  DISCUSS  THIS  WITH  [name2].  HE  IS  OUT  OF  HIS  OFFICE  AT  THIS 

23  TIME  BUT  YOU  COULD  SEND  HIM  A MESS  AND  I AM  SURE  HE  S WILL  GIVE 

24  YOU  ALL  THE  INFO  YOU  WOULD  LIKE,  [operator’s  name] 

25  U:  OK  [operator’s  name].  CAN  YOUPEOPLE  GO  IN  TO  ANY  DIR  AND  DEL  V/  FILES  WHEN  YOU 

26  NOTICE  THAT  THEY  ARE  NOT  BEING  USED.,  [user’s  name] 

27  0:  I AM  REALLY  NOT  SURE,  HOWEVER  I KNOW  THAT  THAT  WWOULD  NOT  BE 

28  RESPONSIBILITY  WE  WOULD  TAKE  UPON  OURSELVES  TO  JUDGE  WHETHER  OR 

29  Y NOT  YOU  WANT  YOUR  FILES  OR  NOT. 

30  [outside  interruption  here] 

31  0:  [user’s  name],  ARE  YOU  THERE? 

32  LINK  FROM  [user’s  name],  JOB  21,  TTY  22 

33  U: SOER 

34  RY  [operator’s  name] 

35  WE  C/  GOT  DIC/SCONNECTED....CAN  YOU  RECOVER  THOSE  FILES  FOR  ME.. 

36  AS  FAR  AS  I KNOW  THEY  E/WR//  WERE  IN  THE  DIR  ON  THE  16TH...THE  NAMES 

37  ARE  ...[file  namel].DAT;lABDLLLL//////////  CHECK  THAT  ...[file  name  1 ].DAT;1,2 

38  [file  name2].F4;l....AND  [file  name3].F4;l [user’s  name] 


I 


39  0:  OK  HOLD  ON  JUST  A MIN.  AND  I WILL  TRY  TO  FIND  THEM 


40  U:  RIGHT 


41  [outside  interruption  here] 


42  0:  ARE  YOU  STILL  THERE? 


43  U:  RIGHT  [operator’s  name] 


44  0:  OK  Y^S  I HAVE  FOUND  THE  FILES  YOU  ARE  CGNECERNED  ABOUT,  ON  THAT 

45  FIRST  ONE  A ASSUME  THE  ABDLLLL  WAS  AN  ERROR  RIGHT? 


46  U:  RIGHT  THAT  SHOULD  HAVE  BEEN  [file  namel].DAT;l  AND  2/ 


47  0:  OK  THEY  ARE  HERE 


48  [outside  interruption  here] 


49  0:  ARE  YOU  THERE? 


50  U:  RIGHT 


• '•"T’r-:™ 
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51  0:  OK  I HAVE  FOUND  THER\R  Y\Y  THE  FILES  YOU  WANT 

52  [outside  interruption  here] 

53  0:  OK  II  HAVE  FOUND  THE  FILES  YOU  WANT  I WILL  RETRIEVE  THOSE  FOR 

54  YOUXXX 

55  YOU  ALSO,  OK 

56  U:  GRA/EAT..WAC  THAT  [name3]  YOU  WERE  TALKING  AV/BOUT? 

57  0:  NO,  IT  G\0  IS  [name4],  HOWEVER  WHEN  YOU  SEND  MESS 

58  SEND  TO  [name2] 

59  U:  RIGHT...IS  [name5]  STILL  AR[outside  interruption  here] 

60  IS  [name5]  STILL  AROUND. 

61  0:  NO  HE  ISN’T.  HE’S  BEEN  GONF  ROR  ABOUT  THREE  WEEKS  NOW. 


62  U:  OK,  THANX  FOR  YOUR  HELP  , [operator’s  name]  HAVE  A GOOD  DAY  [user’s  name]  OUT. 


63  0:  OK  YOU  DO  THE  SAME  WILL  DO  THOSE  FILES  RITHT  AWAY.  BYE 


54  U:  THANX  AGAIN 


r 
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The  transcript  below  is  the  cleaned  up  version. 

10  LINK  FROM  [user’s  name],  JOB  21,  TTY  3 

1 1 U:  HELLO  ARE  YOU  1HERE 

12  0:  YES,  GO  AHEAO 

13  U:  A COUPLE  A QUESTIONS.  ..ARE  THE  FILES  THAT  ARE  ON  DISK  ARCHIVED  EVERY 

1 A WEEK? 

15  0:  YES 

16  U:  IN  ONE  OF  MY  DIRECTORIES  I HAD  SOME  FILES  IN  ONE  DAY  AND  THE  NEXT  THEY  WERE 

17  GONE  AND  THEY  WERE  NOT  ARCHIVED 

18  0:  WELL,  I AM  REALLY  NOT  THAT  SURE  OF  HOW  THE  ARCHIVEING  OF 

19  THE  FILES  ARE  DONE  SO  I WOULD  SUGGEST  THAT  YOU  SEND  A IvESSAGE  TO 

20  [name2]  IN  REGARDS  TO  THIS  PROBLEM.  THE  ARCHIVEING  IS  DONE 

21  DURING  THE  SWING  SHIFT  SO  IN  ORDER  THAT  YOU  DO  DON’T  NOT  GET 

22  MISLED  BY  SOMETHING  I AM  NOT  CERTAIN  ON  I WOULD  RATHER 

23  YOU  DISCUSS  THIS  WITH  [name2]  HE  IS  OUT  OF  HIS  OFFICE  AT  THIS 

2A  TIME  BUT  YOU  COULD  SEND  HIM  A MESSAGE  AND  I AM  SURE  HE  WILL  GIVE 
25  YOU  ALL  THE  INFORMATION  YOU  WOULD  LIKE,  [operator’s  name] 
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U:  OK  [operator’s  name],  CAN  YOU  PEOPLE  GO  INTO  ANY  DIRECTORY  AND  DELETE  FILES  WHEN 
NOTICE  THAT  THEY  ARE  NOT  BEING  USED.,  [user’s  name] 

0:  I AM  REALLY  NOT  SURE,  HOWEVER  I KNOW  THAT  THAT  WOULD  NOT  BE 
RESPONSIBILITY  WE  WOULD  TAKE  UPON  OURSELVES  TO  JUDGE  WHETHER  OR 
NOT  YOU  WANT  YOUR  FILES  OR  NOT. 

[outside  interruption  here] 

0:  [user’s  name],  ARE  YOU  THERE? 

LINK  FROM  [user’s  name],  JOB  21,  TTY  22 
U:  SORRY  [operator’s  name] 

WE  GOT  DISCONNECTED....CAN  YOU  RECOVER  THOSE.  FILES  FOR  ME.. 

AS  FAR  AS  I KNOW  THEY  WERE  IN  THE  DIRECTORY  ON  THE  16TH...THE  NAMES 
ARE  ...[file  name  1 ].DAT;1  ABDLLLL  CHECK  THAT  ...[file  namel].DAT;l,2 
[file  name2].F4;l....AND  [file  name3].F4;l [user’s  name] 

0:  OK  HOLD  ON  JUST  A MINUTE  AND  I WILL  TRY  TO  FIND  THEM 

U:  RIGHT 


[outside  interruption  here] 
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42  0:  ARE  YOU  STILL  THERE? 


43  U:  RIGHT  [operator’s  name] 


44  0:  OK  YES  I HAVE  FOUND  THE  FILES  YOU  ARE  CONCERNED  ABOUT,  ON  THAT 

45  FIRST  ONE  I ASSUME  THE  ABDLLLL  WAS  AN  ERROR  RIGHT? 

46  U:  RIGHT  THAT  SHOULD  HAVE  BEEN  [file  namelj.DAT;!  AND  2. 


47  0:  OK  THEY  ARE  HERE 


48  [outside  interruption  here] 


49  0:  ARE  YOU  THERE? 


50  U:  RIGHT 


51  0:  OK  I HAVE  FOUND  THE  FILES  YOU  WANT 

52  [outside  interruption  hero] 

53  0:  OK  I HAVE  FOUND  THE  FILES  YOU  WANT  I WILL  RETRIEVE  THOSE  FOR 

54  YOU  ALSO,  OK 


55  U:  GREAT..WAS  THAT  [name3]  YOU  WERE  TALKING  ABOUT? 
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56  0:  NO,  IT  IS  [name4],  HOWEVER  WHEN  YOU  SEND  MESSAGE 

57  SEND  TO  [name2] 

58  U:  RIGHT... IS  [name5]  STILL  ARfoutside  interruption  here] 

59  IS  [name5]  STILL  AROUND. 

60  0:  NO  HE  ISN’T.  HE’S  BEEN  GONE  FOR  ABOUT  THREE  WEEKS  NOW. 

! 

} 

! 

61  U:  OK,  THANKS  FOR  YOUR  HELP  , [operator’s  name]  HAVE  A GOOD  DAY  [user’s  name]  OUT. 

62  0:  OK  YOU  DO  THE  SAME  WILL  DO  THOSE  FILES  RIGHT  AWAY.  BYE 

63  U:  THANKS  AGAIN 

64  BREAK 


gum 


.1 
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